Llama 4 (Meta’s flagship ‘AI’ project) was as bad release. That’s fine. This is interative research; not every experiment works out.
…But it was also a messy and dishonest one.
The release was pushed early and full of bugs. They lied about its performance, especially at long context, going so far as to game Chat Arena with a finetune. Zuckerberg hyped the snot out of it, to the point I saw ads for it on Axios.
Instead of Meta saying they’ll do better, they said they’re reorganizing their divisions to focus on ‘applications’ instead of fundamental research, aka exactly the wrong thing. They’ve hermmoraged good researchers and kept AI bros, far as I can tell from the outside.
Every top LLM trainer has controversies. Just recently Qwen (Alibaba) closed off their top base models just to spite Deepseek, so they can’t distill them. Deepseek is almost certainly training on Google Gemini traces. Google hoards their best research for API models and has chased being sycophantic like ChatGPT. X’s Grok is a joke, and muddied by Musk’s constant lies about, for instance, open sourcing it. Some great outfits like 01ai (the Yi series) faded into the night.
…But I haven’t seen self-destruction quite like Meta’s. Especially considering the ‘f you’ money and GPU farm they have. They’re still pushing interesting research now, but the trajectory is awful.
Yes, but its clearly a building block of Meta’s LLM training effort, and part of a pattern.
One implication I didn’t mention, and don’t have hard proof I can point to, is garbage in garbage out. Meta let AI slop and human garbage proliferate on Facebook, squandering basically the biggest advantage (besides cash) they have. It’s often speculated that, as it turns out, Twitter and Facebook training data is kinda crap.
…And they’re at it again. Zuckerberg pours cash into corporate trash and get slop back. It’s an internal disaster, like their own divisions.
On the other side, it’s often thought that Chinese models are so good for their size/compute because they’re ahem getting data from the Chinese government, and don’t need to worry about legal issues.
The research community already knows this.
Llama 4 (Meta’s flagship ‘AI’ project) was as bad release. That’s fine. This is interative research; not every experiment works out.
…But it was also a messy and dishonest one.
The release was pushed early and full of bugs. They lied about its performance, especially at long context, going so far as to game Chat Arena with a finetune. Zuckerberg hyped the snot out of it, to the point I saw ads for it on Axios.
Instead of Meta saying they’ll do better, they said they’re reorganizing their divisions to focus on ‘applications’ instead of fundamental research, aka exactly the wrong thing. They’ve hermmoraged good researchers and kept AI bros, far as I can tell from the outside.
Every top LLM trainer has controversies. Just recently Qwen (Alibaba) closed off their top base models just to spite Deepseek, so they can’t distill them. Deepseek is almost certainly training on Google Gemini traces. Google hoards their best research for API models and has chased being sycophantic like ChatGPT. X’s Grok is a joke, and muddied by Musk’s constant lies about, for instance, open sourcing it. Some great outfits like 01ai (the Yi series) faded into the night.
…But I haven’t seen self-destruction quite like Meta’s. Especially considering the ‘f you’ money and GPU farm they have. They’re still pushing interesting research now, but the trajectory is awful.
This article is about ScaleAI, not Llama
Yes, but its clearly a building block of Meta’s LLM training effort, and part of a pattern.
One implication I didn’t mention, and don’t have hard proof I can point to, is garbage in garbage out. Meta let AI slop and human garbage proliferate on Facebook, squandering basically the biggest advantage (besides cash) they have. It’s often speculated that, as it turns out, Twitter and Facebook training data is kinda crap.
…And they’re at it again. Zuckerberg pours cash into corporate trash and get slop back. It’s an internal disaster, like their own divisions.
On the other side, it’s often thought that Chinese models are so good for their size/compute because they’re ahem getting data from the Chinese government, and don’t need to worry about legal issues.