When Deepseek-r1 was released, everyone was going crazy. People went full doomer mode and called it the “sputnik moment” for the global ai race. I don’t disagree that it wasn’t some sort of an inflection point, but people stopped talking about it after about a week (like most other hyped up things). I think all of these “major” releases are small steps towards the end goal of agi. I’m loving it. Democratized ai is exactly what I’m working towards and that’s why I continue to think that models from labs like meta and deepseek should continue releasing everything open source and moving the needle forward, especially competing with labs like closedai.

A note on the long context window length: this is huge. I predict this will be the big thing of 2025 (at least for the next 3 months) like agentic systems were the hype of 2024. Gemini just released their 2m ctx window length option I think, but that’s nothing compared to 10m. Some people have also said that this is the end of rag and I disagree fully. AI systems need some sort of memory system just like the brain has a memory system. It needs to be efficient in choosing what it stores and precise in what it chooses to recall.

See lex x geohot interview.