• 0 Posts
  • 20 Comments
Joined 1 year ago
cake
Cake day: July 8th, 2023

help-circle


  • Did you read the article, or the actual research paper? They present a mathematical proof that any hypothetical method of training an AI that produces an algorithm that performs better than random chance could also be used to solve a known intractible problem, which is impossible with all known current methods. This means that any algorithm we can produce that works by training an AI would run in exponential time or worse.

    The paper authors point out that this also has severe implications for current AI, too–since the current AI-by-learning method that underpins all LLMs is fundamentally NP-hard and can’t run in polynomial time, “the sample-and-time requirements grow non-polynomially (e.g. exponentially or worse) in n.” They present a thought experiment of an AI that handles a 15-minute conversation, assuming 60 words are spoken per minute (keep in mind the average is roughly 160). The resources this AI would require to process this would be 60*15 = 900. The authors then conclude:

    “Now the AI needs to learn to respond appropriately to conversations of this size (and not just to short prompts). Since resource requirements for AI-by-Learning grow exponentially or worse, let us take a simple exponential function O(2n ) as our proxy of the order of magnitude of resources needed as a function of n. 2^900 ∼ 10^270 is already unimaginably larger than the number of atoms in the universe (∼10^81 ). Imagine us sampling this super-astronomical space of possible situations using so-called ‘Big Data’. Even if we grant that billions of trillions (10 21 ) of relevant data samples could be generated (or scraped) and stored, then this is still but a miniscule proportion of the order of magnitude of samples needed to solve the learning problem for even moderate size n.”

    That’s why LLMs are a dead end.


  • The problem is that there’s no incentive for employees to stay beyond a few years. Why spend months or years training someone if they leave after the second year?

    But then you have to question why employees aren’t loyal any longer, and that’s because pensions and benefits have eroded, and your pay doesn’t keep up as you stay longer at a company. Why stay at a company for 20, 30, or 40 years when you can come out way ahead financially by hopping jobs every 2-4 years?


  • Holy crap, what a garbage ragebait article

    Saving you a click: there’s no new info here, it’s just the same hullabaloo over the guy who made the accusations rescaling the models so they’re the same size, and the author treating it as proof they faked it all

    Which, I don’t personally have a strong opinion on whether it’s faked (especially since it’s been pointed out that models made using different programs and for different platforms can import in drastically different sizes) but it feels kind of disingenuous to say that it’s faked just because of that, y’know? It’s like if an artist takes a 1440p resolution image, traces over it, and posts the traced image in 720p resolution. I wouldn’t consider blowing up the traced 720p to 1440p as “faking” it or altering the traced image.


  • It makes sense to judge how closely LLMs mimic human learning when people are using it as a defense to AI companies scraping copyrighted content, and making the claim that banning AI scraping is as nonsensical as banning human learning.

    But when it’s pointed out that LLMs don’t learn very similarly to humans, and require scraping far more material than a human does, suddenly AIs shouldn’t be judged by human standards? I don’t know if it’s intentional on your part, but that’s a pretty classic example of a motte-and-bailey fallacy. You can’t have it both ways.




  • Who even knows? For whatever reason the board decided to keep quiet, didn’t elaborate on its reasoning, let Altman and his allies control the narrative, and rolled over when the employees inevitably revolted. All we have is speculation and unnamed “sources close to the matter,” which you may or may not find credible.

    Even if the actual reasoning was absolutely justified–and knowing how much of a techbro Altman is (especially with his insanely creepy project to combine cryptocurrency with retina scans), I absolutely believe the speculation that the board felt Altman wasn’t trustworthy–they didn’t bother to actually tell anyone that reasoning, and clearly felt they could just weather the firestorm up until they realized it was too late and they’d already shot themselves in the foot.










  • It’s very user friendly in terms of tooltips, and if you don’t make deliberately bad choices during level up (e.g. taking a feat that gives you a cantrip from the Wizard class… that scales off your INT score… while playing a Barbarian with 8 intelligence that can’t cast spells while raging) it’s fairly difficult to make an unplayably bad character.

    There’s a few cases where some general knowledge of D&D is helpful, such as knowing to never take True Strike because it’s literally worse than just attacking twice and having some knowledge of good builds is useful, since it helps guide what you take when you level up. That said, there’s also entire categories of actions in BG3 that don’t really have an equivalent rule in TTRPG 5e, such as weapon proficiency attacks, so online cookie cutter builds don’t capture the full extent of what you can do.