They can't. The LLMs that underly modern AI systems are auto-regressive, but modern AI systems are not. The "reasoning" systems produce lots of token sequences and pick the best one. That's a form of non-auto-regressive search. What current AI systems do wrong is that they do this search in discrete token space. I've been advocating for inference-by-search for planning/reasoning for many years (before chatGPT). But I think it should be done in abstract representation space. Much of human reasoning/planning is completely unconnected with language. I was correct. You just did not understand my argument.
That's a commonly ignored fact. It's not like Leonardo da Vinci invented the helicopter, or even Paul Cornu who built one that took off in 1907. But Louis Bréguet invented the swashplate, and Henrich Focke flew a controllable helicopter with contra-rotating rotors, all before Igor Sikorsky came up with the variable-pitch tail rotor and made the whole thing practical in 1939. Getting the basic idea is the easy part. Building it is harder. Making it work is even harder.
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI" That statement is still totally true. First, the reasoning abilities of current AI systems are based non-auto-regressive search (which is what I have always advocated for). But AFAICT, they do it in token space, which is limited and inefficient. I have claimed that human-like reasoning must be a search in continuous representation space. It looks like the industry is moving towards that. Second, the self-improvement methods, as currently practiced, only work for domains where the quality of outputs can be scored without human intervention, such as mathematics, code, and scenarios that can be simulated accurately. Not anything else. Humans and animals learn new skills way more efficiently than current RL methods. Third, the multimodal capabilities of current AI assistants generally use separately-trained encoders (that are not LLMs). This is also what I've been advocating. Except that I think the best way to do this is with JEPA trained with self-supervised learning. The research community is clearly moving towards that (3000 papers on JEPA in just 4 years). Fourth, if LLMs were a path to human-level AI, we would have domestic robots and Level-4 or Level-5 self-driving cars for consumers by now. And we don't. We certainly don't have cars that can learn to drive in 20 hours or practice like any teenager. We're still missing something pretty huge to claim human-level intelligence (let alone superhuman). Sure, we now have computer systems that are impressive, very useful, and whose performance is superhuman in an increasing number of domains (coding being one of them). But that's true of the entire history of progress in computer technology. Lastly, there is a basic confusion about what intelligence actually is. It is not the mere accumulation and regurgitation of existing declarative knowledge (which is essentially what LLMs do). As Jean Piaget famously said, "intelligence is not what you know, it is what you do when you don't know." It is your ability to solve new problem without any prior training, to act in previously-unknown scenarios, and to adapt very quickly to new situations with minimal training. We're still far from that.
I’m very excited by this new benchmark dataset for visual generation that is suitable for the modern era of large scale generative models!🤩 Keshigeyan Chandrasegaran@keshigeyan 1/ Introducing GPIC: a Giant Permissive Image Corpus and benchmark for visual generation! 🚀100M VLM-captioned image-text pairs for training 📊1M image-text pairs for benchmarking 🖼️~28 trillion pixels 🤗Centrally Hosted ✅Fully permissive for research + commercial use Dataset, benchmark and models🧵👇 Co-led with @KyleSargentAI 🔗 View Quoted Tweet
Plot twist: I was Vapnik's Department Head between 1996 and 2002, and made sure he had the space, resources, and collaborators to do his best work. To people who think I'm dogmatic: I famously disagreed with Vapnik about the best way forward in ML, but did what I could to promote his work, because research is about exploration and science is about results.
@ilyasut To prevent models from going rouge, just make the neoclouds go green. Or threaten to blacklist them with a yellow card or a pink slip. Preventing them to go rogue is another story, probably involving guardrails.
I would try to figure out why LLMs can write my essays but not clean my bedroom. Then I would study topics in college and grad school that could help solve that problem. I'll figure out a set of methods and architectures beyond LLMs that can quickly learn to perform physical tasks as efficiently as humans and animals. That last item is also what I would if I were 30, 40, 50, or 66 years old 😉
New course: Build AI agents that generate images and videos -- an under-explored frontier. A key to performance is having the agent evaluate its own output, and iterate to improve quality. This short course is built together with @googlecloudtech and taught by Katie Nguyen and Wafae Bakkali. You'll learn three evaluation techniques and combine them in an agent: image-text similarity scoring to check the output matches the prompt, an LLM judge that scores against custom criteria like brand consistency, and structured rubrics that break a prompt into verifiable yes/no questions like "is the subject in the frame?" and "does the camera motion match?" Skills you'll gain: - Learn image and video prompt engineering - Build an image agent that turns brand guidelines into UI mockups - Build a video agent that plans multi-scene explainers and animates reference frames with synchronized audio Join and build agents that create images and video! deeplearning.ai/courses/ai-age… Your browser does not support the video tag. 🔗 View on Twitter
I’m very excited by this test time training work for robotic learning! It’s an awesome collaboration between @StanfordSVL and @NVIDIARobotics ! Jim Fan@DrJimFan We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 sec), instantly forgetting what just happened. We pushed to 3 orders of magnitude beyond SOTA. Introducing RoboTTT. Test-Time Training (“TTT”) carries a tiny model *inside* the model. Every incoming sensor reading triggers one gradient step on that tiny core, so the history keeps getting compressed into its weights. The hidden state has a fixed size (literally a small neural net), so the robot can “grok” arbitrarily long experience with little overhead. Learning continues indefinitely after deployment. We can then put an entire video in context as prompt! RoboTTT enables one-shot in-context learning from human video: in circuit board assembly, a human demonstrates a never-seen configuration once, and the robot imitates it faithfully. Humans drop things all the time, but we pick them up so fast that we don’t even notice. That reflex to fix is half of our physical competence. RoboTTT shows self-improvement on the fly: the robot is skilled at recovering from its own errors mid-episode, and each fix enters its context to inform the next move. The TTT core distills a general-purpose, failure-to-correction mapping from the training data. One more thing. What excites me the most is a new Context Scaling Curve: from 128 to 8K timesteps, closed-loop performance hill-climbs steadily with no sign of saturation. 8K-context pretraining beats 1K by 62%. What LLM enjoys, robotics should too. Soon, even 1M context is not a fantasy. Deep dive in thread: Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
The new White House policy requiring green card applicants to apply from outside the US is a capricious attack on legal immigration. It will hurt families, leave us with fewer doctors, teachers and scientists, and hurt American competitiveness in AI.
@BrianRoemmele @elonmusk The Medal of Science is for scientists. Scientists publish their works in peer-reviewed venues so they can be scrutinized, verified, and reproduced.
Remember October 2022 when you doused Galactica with vitriol? Galactica was a 120b-parameter LLM-based system from Meta-FAIR designed to help scientist write papers. It was open sourced (link below). A mob of haters, including Michael, claimed it was dangerous and toxic and was going to destroy Science. The small team at FAIR couldn't sleep at night and took down the demo website (they kept the GitHub and paper up). Then, only 3 weeks later, ChatGPT was released and was welcomed as the second coming of the Messiah🤔 The vitriol dousers were silent. technologyreview.com/2022/11/18/106… k github.com/paperswithcode… L
Thank you for this, @_NathanCalvin . I appreciate and am a fan of the investigative work you’ve done on behalf of AI safety and believe your comments are coming from a constructive, honest place, and that it benefits the public to have people whose job is to scrutinize the actions of frontier AI companies. I think something would be wrong if companies of the size of Anthropic didn’t get harshly criticized basically every day; strong antibodies against corporate power are a healthy feature of a free society, and especially so when issues of the gravity of the one we’re facing are at play. I also do agree that some of the examples you cite were mistakes by Anthropic. That said, I disagree with the thrust of your post. At the core, a good fraction of the things you criticize are consequences of Anthropic’s basic theory of change that we have held for many years. Our strategy has always been what I’ve called the “race to the top” — by showing how it’s possible to build the technology responsibly, we can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier; that’s one lesson I’ve learned repeatedly. And looking at that strategy as a whole, I’m proud of our track record and stand by it. I believe that if we take a list of things that Anthropic did in the past year that were pro-safety and commercially costly to us, they far outweigh the list you’ve made, while at the same time I also disagree with some of the items on your list. Here are some things Anthropic has done, in the same period you’ve cited (the last year), which were very costly to us commercially while having strong safety benefits: 1. We played a key role in SB53 (and other state laws), at a time when doing so was politically perilous to the company and AI regulation was very unpopular. We in fact argued for a stronger version of SB53 than was ultimately passed; there were a lot of stakeholders and we were not the limiting factor in the strength of the bill that was passed. 2. We have very strong biological safeguards on our models, because we’ve seen concerning results in bio uplift trials and even seen and blocked potential attempts to misuse our models for bioweapons (as seen in our threat intelligence report from a few weeks ago). This is a very serious threat that we want to have zero tolerance for. But having very low false negatives means having significant false positives (despite months of effort on our part to improve the classifiers), enough so that biologists joke about how restrictive our classifiers are, and competitors sometimes sell against us by touting their more permissive bio capabilities. This has been very painful to us in an area where we care a lot about supporting an ecosystem of Claude users (biomedical science; and it is particularly frustrating to me as a biologist), but it’s the right thing to do and we intend to keep doing it. 3. Similarly, we first had Mythos-class models in February but didn’t make them generally available until June (when we had stronger cyber safeguards in place). We only offered them to a limited set of partners in order to focus on cyberdefense. It would have been much more economically rational to simply release the model to everyone. We likely sacrificed a significant chunk of a several-month model lead. But because of this, many companies secured their cyber infrastructure before the deluge of widely available cyber-capable models. Of course there are ways it could have gone better or been run better, but this was what we were aiming to do. 4. I advocated forcefully and publicly, in my essay Policy on the AI Exponential in June of this year, for the government to have the power to block model releases. Back then this was still outside or maybe on the edge of the Overton window, though I understand the window has moved since then. W
It's a companion to Machines of Loving Grace, an essay I wrote over a year ago, which focused on what powerful AI could achieve if we get it right: darioamodei.com/essay/machines…
@ziv_ravid @chrmanning Sooo where is my domestic robot? Where is my L5 self-driving car? I can fine-tune an LLM to answer any question about the physical world. But that doesn't mean it has enough understanding and physical intuition to predict what a 6 year-old can predict.
I have absolutely never ever "asked" to stop R&D on LLMs. In fact, I was very supportive of LLM efforts at Meta. Just look up my public advocacy for Galactica (November 2022), and the open sourcing of OPT-175B (May 2022), and of the Llama herd (mid 2023). I was actually influential in convincing the Meta leadership to start an AI-first product division built around Llama in January 2023. I've always said that LLMs were impressive, useful, and had amazing potential. I've also always said they were an off-ramp on the path towards human-level AI. Stop making me say things I didn't say or claim that I somehow hindered LLM R&D at Meta. That's BS.
Our latest work in closing the last-mile gap on long horizon robotics tasks. Wensi Ai@wensi_ai Long-horizon, whole-body robot policies still fail during deployment. How do we best close this last-mile gap? 🤖 🚨 Introducing I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation Website i-3l.github.io Oq Paper arxiv.org/abs/2610.17748 bI Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
The world of R&D is forking into two paths: the token-abundant research, and the token-starved research. The future is in the former - evidentially, the progress by today's top AI industry teams and neolabs is breathtaking, where researchers’ human brilliance is super charged by AI’s assistance. Every research university president should be reading this report and reflecting on what the future of higher education research should be. openai.com/index/research…
The goal of building any technology, AI included, should be bettering human lives and society. Bloomberg TV@BloombergTV “Any threat to human society, including existential, is within ourselves,” says World Labs Technologies CEO Fei-Fei Li as she discusses the risks surrounding AI, and the responsibility humans have in shaping how the technology is developed and used. Listen to our full interview here: bloom.bg/3V74HgP Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
“Loop engineering” is a hot buzzphrase after mentions of it by Boris Cherny (Claude Code’s creator) and Peter Steinberger (OpenClaw's creator) went viral on social media. Loops are now a key part of how we get AI agents to iterate at length to build software. In this letter, I’d like to share my 3 key loops, shown in the image below, for building 0-to-1 products. These loops guide not just how I build software, but also how I decide what software to build. Agentic coding loop: Given a product specification and optionally a set of evals (that is, a dataset against which to measure performance), we can have an AI agent write code, test its work, and keep iterating until the code is bug-free and meets its specification. This idea of closing the loop took off around the end of last year, and it has been a game changer in enabling coding agents to work longer productively without human intervention. For example, over the weekend, I was building an app for my daughter to practice typing, and my coding agent could easily work for around an hour, using a web browser to check what it had built multiple times before getting back to me, without needing my intervention. The engineering loop executes quickly. Every few minutes, the coding agent might build and test a new version of the software. I hear frequently from developers who are finding new ways to engineer more effective engineering loops. This is an active area of invention! Developer feedback loop: In this loop, a developer examines the current product and steers the coding agent to improve it. Last year, a lot of developers (including me) were acting as the QA (quality assurance) function for our coding agents, manually finding bugs and then asking the agent to fix them. But with coding agents much more able to test their own code, the amount of time we need to spend on this function has decreased significantly. This allows us to make higher-level product decisions, such as what key features to offer, where the UI needs improvement, and so on. The developer-feedback loop operates over time intervals between tens of minutes and hours — that's how frequently a developer might review a product and give feedback. In the case of the typing app, I changed my mind a few times about the visual design, what cat costumes she can unlock as she learns (she loves cats), and the user flow for a grown-up to log in and steer the child's learning experience. When a developer has a clear vision for what to build, it is still a lot of work to translate that vision into a specification for a coding agent to implement. Further, after the developer has seen an implementation, they might update (or perhaps clarify) the spec to steer it toward what they want. If you find that the system repeatedly runs into certain problems, building a set of evals for the agent becomes useful. AI-native teams are increasingly using AI to help shape product direction, for example, automating the gathering and analysis of usage data, summarizing written and verbal customer feedback, or carrying out competitive analysis. However, for pretty much all the products I’m involved in, I see humans as having a significant context advantage over current AI systems — we know a lot more than the AI system about the users and the context the product has to operate in — and thus humans play a critical role. Many people describe this human contribution as “taste,” but I prefer to think of it as humans having a context advantage, since that gives us a clearer path to helping AI systems get better. This also speaks to why this step can’t be automated: So long as the human knows something the AI does not, human-in-the-loop is needed to to inject that knowledge into the system. External feedback loop: This includes a wide range of tactics like asking a few friends for feedback, launching to alpha testers, or putting the code into production with A/B testing. These tactics are usually slow
The 1991 version is an obscure tech report with a vague idea, no theory, no experiment to speak of, and certainly no code, that the author never revisited or pushed. It has one hand-drawn architecture diagram, one formula of the type f(x)=0, and a claim that every other technique is a special case if you instantiate f and x properly.
In addition to transparency, I now believe frontier models should face mandatory third-party testing for cyber, bio, and autonomy risks—with the power to block or revoke deployment of models that pose catastrophic risk.
Announcing OpenWorker! An open-source agent that doesn't just chat with you, but delivers finished work -- like hand you a polished document, send a slack message, or update a calendar entry. Ask it to prepare a customer brief, untangle your calendar, draft a report, or triage a Slack alert. It works across your files and everyday tools, produces the deliverable, and checks in before doing anything consequential. OpenWorker runs on your Mac, with Windows support coming soon. It does not lock you into any one model. Bring your own API key and run it with GPT 5.6 Sol, Claude Fable, Gemini 3.6, an open weight model (like Kimi, GLM, DeepSeek, Inkling), or Ollama to keep your data local. Your data does not leave your machine except through an LLM provider and integrations that you choose. @rohitcprasad and I are building OpenWorker because AI coworkers are an important way to get work done, and we want there to be an open, privacy-preserving, model-independent option. Check it out and let us know what you think! Try it out: openworker.com (requires your own API key) Source code: github.com/andrewyng/open… Your browser does not support the video tag. 🔗 View on Twitter
Good move by @JensenHuang . The Nvidia letter is well written and worth reading. As we saw with the OpenAI-Hugging Face hack, we need open models and harnesses for defense. Lets stop believing the PR that closed models are safer. - that's just regulatory capture. Jensen Huang@JensenHuang Attackers have frontier AI. Defenders need a frontier AI ecosystem—the best open and closed models, force-multiplied by a global community. During the Hugging Face incident, closed AI blocked essential forensics. An open-weight frontier model helped contain the intrusion. That’s why we created the Open Secure AI Alliance. 🔗 View Quoted Tweet
Congratulations @JeffDean , @Sanjay_Ghemawat , @OriolVinyalsML and @quocleix ! Knowing you guys, this will be amazing. Cheering you on and wishing you well on your important work! Jeff Dean@JeffDean Announcing Discovery Loop! I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat , @OriolVinyalsML and @quocleix , we are founding Discovery Loop ( @DiscoLoopAI ), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor. ♾ Learn more at: discoveryloop.com 🔗 View Quoted Tweet
Today we announced the Claude-led discovery of a molecular machine that we suspect could represent a new gene editing mechanism. Its precise function, biotechnological utility (if any), or level of significance is not yet clear, but at minimum it is work I would have been proud to do as a PhD student. The work was done mostly, though not entirely, by Claude: our life sciences team suggested a broad area of research, Claude read through the literature and a bunch of genome data and discovered something interesting, then Claude proposed experiments to verify the discovery and our team carried them out. It’s easy to dismiss this as a one-off or curiosity, but we’ve repeatedly seen a pattern where AI performance in new intellectual domains goes from weak to superhuman in a matter of a few years. In 2023 models struggled to do math at the level of an average high-school student. In 2024 they started to do well on math competitions for the best high-schoolers in the country, in 2025 they started to solve minor open problems, in early 2026 more significant open problems, and in late 2026 they are beginning to solve the top few open problems in all of mathematics. We believe AI for biology is on a similar exponential trend. The main difference between biology and mathematics, of course, is that math can be done purely theoretically, while biology requires experimentation. Some have used this to draw the conclusion that AI’s utility in biology will be limited. We think this is wrong. As we’ve demonstrated today, humans can collaborate with AI to perform the experiments, validate key results in a few weeks and, if necessary, work with the AI to iterate on what they find. Eventually it may even be possible for Claude itself to safely perform the experiments by autonomously controlling lab equipment, with appropriate safeguards in place, but we aren’t doing that today (our lab is also a BSL1/BSL2 facility that doesn't handle materials dangerous to humans). More broadly, biomedical advancement has many stages — from fundamental biology discoveries, to translational research, to drug discovery, clinical trials, and finally the actual delivery of medicines and health care to patients. We are also interested in these later stages, but even simply accelerating the first stage of fundamental biological discoveries has the potential to speed up and broaden the entire pipeline. Improving our understanding of biology and sharpening biologists’ tools can drive forward all of the later stages, for example by identifying new drug targets, finding new therapeutic modalities, allowing for more precise measurement, and speeding up the experimental loop which itself further accelerates our understanding of biology. This will not in itself speed up clinical trial times, but if it succeeds it could greatly increase the number of promising candidates that go into the pipeline — an increase in throughput even though latency remains. In Machines of Loving Grace, I wrote about AI’s potential to “cure most diseases in 5-10 years” — a goal that sounds impossible, but one I believe is just barely possible if AI is applied to every stage of the pipeline. The first step is showing that AI can first help with, and then drive, biological discoveries. Claude’s discovery is the latest in a line of related prior work that goes back decades, beginning with systems like CRISPR, and continuing with discoveries like the bridge recombinase and VIPR in the past few years. Recently, there has been heightened interest in systems based on reverse transcriptase (RT) enzymes, the enzyme underlying the system Claude identified. And most recently, a Stanford team working independently described a novel RT system with an associated non-coding array that is in some ways similar to the one Claude found, though they are distinct systems that evolved independently from each other. I believe that we’re at the very beginning of finding such systems and de
@firesidealpha Sudden Clarity Sam realizing that real-world dynamics equations have inertia and friction terms. (Particularly when they concern large systems like human societies). If only the singularitarians, AI doomers, AGI-is-near crowd, and other SV cultists could realize that....
Sim-to-real aligned simulation also makes evaluation scalable. Here, the same failure behavior and outcome appear in both virtual and physical worlds. The simulation captures more than the task setup or final success label; it reproduces the conditions that push a policy toward success or failure. The result is faster iteration, broader coverage, and substantially lower cost. Your browser does not support the video tag. 🔗 View on Twitter
@BrianRoemmele It's as if my electronic wind instrument could stop me from playing along Miles Davis or John Coltrane solos while I'm (painfully) trying to learn from them. Anthropic is insane and insufferably moralizing. But we already knew that.
Tired of winning Steve Rattner@SteveRattner Oil is back above $100, and gas is above $4. These numbers are brutal for inflation and Americans' pocketbooks. And there is no end in sight for their fundamental cause: Trump's war of choice in Iran. 🔗 View Quoted Tweet
Incredible opportunity for robotic learning researchers/engineers to join @theworldlabs ! ❤️🔥 Yunzhu Li@YunzhuLiYZ We're hiring in robot learning at @theworldlabs ! Join me, @drfeifei , and the team to define and scale the next generation of world models for robot learning! Atlas for Robotics: worldlabs.ai/blog/atlas#rob… Real-to-Sim-to-Real: worldlabs.ai/blog/real-to-s… Apply: jobs.ashbyhq.com/worldlabs/8599… Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
6.1 Terra is here! It is good at a lot of things, but we are especially excited for this new era of computer use. OpenAI@OpenAI GPT-6.1 Terra: delivering fast and capable computer use at a lower price. GPT-6.1 Terra understands what’s on screen, takes action across apps, writes naturally, and is a more capable model for your whole workday with the efficiency to take on more of it. Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
@Kasparov63 Several disciplines are already akin to "gyms of the mind" That includes intellectual games, such as chess and go, and some areas of pure mathematics. The best at these activities are the counterparts of top athletes for physical games.
For about 10 years now, I have argued that the *only* way forward is for AI technology to be widely available, shared, and open. Like the printing press and the Internet, AI amplifies human intelligence and efficiency by improving access to knowledge. To empower individuals, societies require a high diversity of AI systems with different value systems, linguistic abilities, philosophical/political biases, and specific expertise. We need diverse AIs for same reason we need a diverse press. Given the cost and complexity, this can only be achieved through open foundation models on top of which anyone can build systems with their languages, biases, expertise, and value systems. I have been more vocal about this over the last 4 years, since AI popped into the public discourse. I have made the argument in various forums: corporate C-suites, AI safety discussion groups, professional meeting, the US Senate, the UN Security Council, and the public sphere through media interviews, podcasts and social media posts. I totally agree with @finkd Mark Zuckerberg's recent piece in which he writes: "the notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic. Historically, hoping that an absolute power will benevolently provide for humanity if sufficiently enlightened has not led to safe or positive outcomes.” When @DarioAmodei writes: “some may object that we can simply keep AIs in check with a balance of power between many AI systems, as we do with humans", he is talking about me, among (thankfully) many others. It is the only good path forward. There will be nefarious uses of AI, as there have been with every technology ever invented. But it will be your Bad AI against my Good AI.
When you work on research (particularly on long-term research), you have to accept the fact that corporate management, who is often interested in short-term impact, may think you are wasting your time and company resources. But it very much depends what you objectives and your horizon is. - If your goal is keeping up with the current AI product race, then you might think that long-term bets are a waste. - If your goal is "AGI" (whatever you mean by that) then your horizon is likely longer and your modus operandi is very different. - However, If you think "AGI is just around the corner through LLM scaling", then you might think working on world models and other new AI concepts is a waste. - But if you know that AGI requires a few more conceptual advances (as Demis and I do), you pick a role that maximizes progress towards that goal. That role is necessarily less connected with products, less connected with management, and more connected with research. - I can't speak for Demis, but in my case, I've always been supportive of high-quality work on LLMs and impressed by the results. My personal interest was to advance towards human-level AI, and I never thought LLMs were the ticket. But I always thought LLMs were incredibly useful as a technology, and eventually as the centerpieces of a family of products. As a high-level IC (not a manager), I would not have had the influence or power to stifle work on LLM even if I had wanted to (which I had not).
OpenWorker -- an open source agent that doesn't just chat but completes tasks on your laptop -- just released a new version with many features for security workflows. After our initial release, many users found it especially useful for cybersecurity. Attackers are already using AI; OpenWorker is committed to giving defenders the same leverage. Running an agent requires both (i) A model and (ii) A harness (the software around the model). Because the OpenWorker harness is fully open source, security teams can audit it to make sure we haven't built any backdoors that exfiltrate your code and data to some company or even a foreign adversary. OpenWorker now comes with built-in cybersecurity agents for (i) Scanning your code for vulnerabilities. (ii) Scanning dependencies for supply chain injections. (iii) Checking your cloud security configuration for attack surfaces. This enables developers to do much more security work before deployment (part of what's called the "shift left" movement). You choose the model: you can run open weight models fully locally so sensitive code never leaves your machine. This helps with legitimate security work (like reproducing a known exploit to defend against it) that can trigger refusals in leading closed models. Or use your ChatGPT subscription, or stealth preview models like Ox Alpha, or any model via API key. Thanks also to all the open source contributors! Join work with @rohitcprasad so please follow him too to get more frequent updates. Try it out: openworker.com Code: github.com/andrewyng/open…
Tired of not winning Republicans against Trump@RpsAgainstTrump Donald Trump “erupted” during a meeting with his top national security team last week, reportedly yelling expletives at those present, according to NBC News. The report says Trump is “exasperated” by the lack of progress in ending the Iran war and the lack of a clear strategy. One Trump ally said, “The president is exasperated. I don’t think he believed it was going to be this difficult to get the Iranians to agree to a deal.” 🔗 View Quoted Tweet
@YunzhuLiYZ , @fast_sploosh , and @xhsonny have built a remarkable team that is training and evaluating robots in high-fidelity simulation – proven not only in lab demos, but in live deployments on real hardware. Last month, we wrote that the boundaries between rendering, simulation, and planning are beginning to blur. Bringing our world models together with SceniX's simulation and robotics expertise is an important step in our quest for spatial intelligence. Welcome to the team. Read the full announcement: worldlabs.ai/blog/scenix
There is some speculation about our partnership with Cerebras. Cerebras is a close partner, and we have a deep engagement pushing on the frontiers of speed.
Progress in airplane design have made them safer and safer. Airplanes would have been safer earlier with *accelerated* progress. Turbofan engines have become so reliable that long-haul airplanes can now have only two of them. The same is true for AI: better AI is safer AI. "Pacing", whatever one means by that, is counterproductive.
Thank you Mark, Alex and the whole Meta team for your contributions to open weight AI. Mark Zuckerberg@finkd Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases. Congrats to @alexandr_wang and the MSL team for all your great work on these models. 🔗 View Quoted Tweet
Join me and 2000 other AI engineers and builders at AI Dev in New York City, Nov 30-Dec 1! A LinkedIn study found that over July-September, the job title AI Engineer has been one of the 10 fastest-growing occupations in seven major economies. AI engineering has become a core skill for all software work, and AI Dev is designed to help you develop it. Our sessions give you hands-on practice with: - Coding agents and harness design: Harness tuning, repository context, autonomous coding - Agent memory and knowledge graphs: GraphRAG, vector search, persistent memory, hybrid retrieval - Agents in production: Evals, observability, simulated testing, browser and voice agents @ylecun and I will also talk about what’s next in world models and physical AI. I hope to see you there! Tickets: ai-dev.deeplearning.ai
Adam Thierer had written the landmark book on "Permissionless Innovation." We get the best ideas when we don't have to ask the government in advance for permission to invent. Protecting open source AI is now a critical part of ensuring this. Adam Thierer@AdamThierer We stand at a critical crossroads in the debate over AI governance in the United States, and it feels like we are inching closer to a very serious battle over whether or not open source models will even be allowed in an environment where a new de facto licensing regime has been taking shape. Lacking formal congressional statutory frameworks or clear administration rules (like the diffusion rule revision), we appear to be left with a sporadic, arbitrary, non-transparent process for model review. The fiction of “voluntary” agreements hangs over this debate, and some large model developers are already showing an incredible willingness to bend over backwards to accommodate national security-related officials / orders that the rest of us are not privy to. It's a very opaque process. And those model developers are expected to play ball with those officials, or else their models get pulled from the market or held up for long periods. Or they will lose any government procurement contracts they have. There is nothing “voluntary” about it when that Sword of Damocles hangs in the room. As this mess worsens, at some point the question of how to handle open source models will come into sharper focus because it will have to. I've even heard some rumors lately that something may be coming from the admin on this front to address this. Needless to say, if this informal new AI model review regime expands and takes on more pre-vetting characteristics / requirements, it is hard to see how open source players could comply with such quasi-licensing of AI models. Specifically, if this ambiguous new regime is accompanied by a general presumption of ‘restrict-until-permitted,’ then that would spell doom for open source. That is a very dark path for our country. Worse yet, of course, would be a move by national security officials to more directly restrict open source models and capabilities. If that happens, then we would be right back in the thick of a Clipper Chip-like battle along the lines of what we saw in the late 1990s. That is a much darker path for America. Meanwhile, open source developers have no “golden shares” or other goodies to offer the government to make their problems go away. Let’s be clear: If our government takes the dark path, it will become the single most important battle over computational freedom of modern times. It is time for people to make a stand in defense of open source before it is too late. 🔗 View Quoted Tweet
New course: Build LLM applications that respond to user requests quickly by running on hardware designed for fast inference. This short course was built with @Cerebras and taught by @zhennydez , @duerr_seb , and @MilksandMatcha . When a model generates text, much of the time is spent moving its weights out of memory and into the compute units. Inference-optimized hardware minimizes that movement, making token generation several times faster than on a typical GPU setup. In this course, the hardware you'll use is Cerebras' Wafer-Scale Engine, which is designed for fast inference by keeping the model's weights close to the compute units. Fast inference makes lengthy agentic workflows go faster, and also unlocks latency-sensitive, real-time applications like live translation and voice agents. Skills you'll gain: - Compare how GPUs, TPUs, and Cerebras' Wafer-Scale Engine each handle the memory-to-compute bottleneck - Build real-time applications powered by fast inference, including personalizing a webpage and running a multi-step workflow to analyze market signals - Adopt concrete habits for agentic coding with fast inference, keeping your sessions focused and steering the model more effectively My teams use Cerebras for several applications that are latency sensitive. Join and build LLM applications that respond quickly: deeplearning.ai/courses/fast-l… Your browser does not support the video tag. 🔗 View on Twitter
But there is a 1 hour talk, with results and evidence, that leads to these conclusions. Also, I am not saying that LLM-based AI tools are useless. They are very useful. We all use them. But, by themselves, they are not a path to human-level AI. Architectures that understand the real world are not LLMs. The architectural components that enable current LLM-based AI systems to interpret images, videos, and other real-world signals are not LLMs.
@abuchanlife - training frontier models is expensive - distilling a frontier model is cheap - this market force alone tells us that frontier foundation models will be free/open. - there are others. thealliance.ai/projects/tapes…
The creativity and imagination is out of the world! So grateful that @theworldlabs got to partner with the amazing talents @withloreco to translate their incredible ideas into an interactive experiences for users to enjoy!🤩 World Labs@theworldlabs We turned dreams into worlds. Then filled them with history's greatest minds. Not a video. A world, running directly in your browser. Step inside ↓ Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
Nope. It's still true. Where is your domestic robot? Where is your Level-5 self-driving car? Where is your robot car that can learn to drive in a few hours of practice like any 17 year old? Where is your AI system that can understand the real world and quickly learn new skills like a house cat? There is no question that AI will eventually become as intelligent as humans in all domains. *** BUT *** 1. We're still far from that, even if AI and computer technology surpasses humans in an ever-increasing number of tasks. 2. It won't be based on LLMs, although LLMs will have a role to play (e.g. as a text interface).
@jestermolecule @HarshithLucky3 but only because...i actually do want the world to have tons of memory and people to put as much as they want in their computers, and it is fucking annoying that this was taken out of context. i have been arguing for the world to ramp up memory production for a long time.
I'm definitely not anti-achievement and never criticized real achievements, including by Elon. I even own two Teslas. I do expose marketing stunts that, while cool on the surface, are not the achievements they are claimed to be. Compressed air thrusters might give you acceleration boosts in any direction. They might even make your car hover for a couple of seconds if you accept replacing the passenger seat by a 400 liter high-pressure air tank. But they are not a path to actual flying cars. The same way Burt Rutan's SpaceshipOne was never going to be a path to orbital flights (just bunny hops out of the atmosphere). I am anti-Musk when it comes to politics. His political actions, the massive dissemination of disinformation through X and other channels, and his support for authoritarian regimes and racist ideas have the effect of weakening democracy and weakening human decency and social progress around the world.
@gabriberton You obviously did not understand my statement. I was talking about auto-regressive token prediction, which is what pure LLMs do. But good code generation systems aren't pure LLMs and aren't doing mere auto-regressive token prediction.
I am very grateful Aleksander accepted our request to come take this on; the safety and security challenges in front of us as we confront superintelligence are immense, and he has an excellent set of skills to work on this. Aleksander Madry@aleks_madry Personal update: Turns out that what I thought was leaving OpenAI became more of a sabbatical. When I left a few months ago, I felt that the most important thing to focus on was making AI more useful to people. But we’ve since entered a different regime of AI progress. One in which getting the safety and security challenges of increasingly capable AI right has to become a central priority. So I’ll be returning to OpenAI, joining @merettm to help chart a path through these challenges. There’s much work to do, so let’s get going. 🔗 View Quoted Tweet
Glasswing is just the first step: patching and securing the world’s software infrastructure will be the work of months and years, and will require even broader cooperation across AI companies, cyberdefenders, software providers, governments, and more.
The amount of basic research done in industry is nowhere near the amount done in universities. At various times, there have been industry labs that had fundamental research activities and have made important scientific contributions. Examples in information technology include Bell Labs, IBM Research, Xerox PARC, GE, Phillips, NEC, and several other. That disappeared in the 1990s. Microsoft Research picked up the torch in the 2000s, followed (to some extent) by Google and then Meta (for about a decade until recently). But their innovations almost always built on top of academic work, and certainly profited from the whole research ecosystem.
Nice! This ends up being a version of what some of us have called "target prop": every layer's input is a free latent variable that serves as a target for the previous layer. As this paper points out, this can be derived from an "augmented Lagrangian" formulation of backprop in which the constraints (input of layer k+1 = output of layer k) are turned into penalties (divergence between input of layer k+1 and output of layer k). I've always hoped more people would pick up on this idea. I'm happy this is happening! I must say though that target prop, in the end, optimizes the same criterion as backprop and does the same thing as backprop while evaluating the gradient in a different way, perhaps more biologically plausible. My lab did some work on this idea in the context of "sparse auto-encoders" in the late 2000s. It turns out when the code in an auto-encoder is regularized (e.g. with L1 to make it sparse) target prop seems more efficient than backprop. scholar.google.com/citations?view…
Question for everyone : Are you pro-intelligence or anti-intelligence ? Do you think more intelligence in the world is intrinsically good ? Or do you think more intelligence is intrinsically dangerous ? AI amplifies human intelligence. It makes PEOPLE smarter and more effective. It's like search engines, libraries, education, literacy, language: it makes PEOPLE smarter and more empowered. If you want intelligence (artificial or natural) to be under control of a few companies or governments, you are a neo-obscurantist or a neo-feudalist.
Au contraire. C'est une nouvelle ère qui s'ouvre pour les mathématiques. Une ère où la démonstration formelle est largement automatisée et où l'accent sera reporté sur le développement de nouveaux concepts, nouvelles abstractions, nouvelles définitions, et nouvelles conjectures. L'invention du bateau a réduit l'importance de la nage, mais a permis la découverte de nouvelles terres.
Launching something new with Shaun Johnson and Oriol Vinyals: A deeply technical VC firm focused on early stage AI startups. My main gigs are still with AMI - Advanced Machine Intelligence and the NYU Courant Institute School of Mathematics, Computing, and Data Science. But 224 Ventures connects us deeply with the AI startup scene through our network of founders and amazing LPs.
You should be able to use your AI subscription wherever you need: Vignesh@vigyso I am going to try and answer a bunch of questions that people have raised about Sign in with ChatGPT in the next couple of days. Before that, I want to let you all in to why we shipped Sign in with ChatGPT (SIWC) in the first place. Clarify our intentions here before getting to the specifics. 🔗 View Quoted Tweet
The most important skills for using AI coding agents effectively. Presenting the AI Engineering Skills Map for using coding agents. x.com/i/article/2095…
The Real-to-Sim part of R2S2R transforms physical robots, sensors, environments, objects, and interactions into simulations that preserve task-relevant observations and dynamics - not only how the world looks, but how it acts when the robot interacts with it. These results set a new bar for sim-real alignment in robot manipulation. Your browser does not support the video tag. 🔗 View on Twitter
ChatGPT can now generate a custom UI for you! OpenAI@OpenAI GPT-6 and Intelligent UI, now rolling out in ChatGPT for everyone. Intelligent UI in ChatGPT delivers fast, interactive answers that make everyday questions more visual, complex topics easier to grasp, and interactive tools for your task available on the spot. Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
The loudest voices stoking fears about AI dangers have made tremendous headway in the past two weeks. AI technology has not taken some unexpected, dangerous turn, but the hype around it — propelled by what appears to be a well orchestrated PR campaign — has drummed up considerable fear. I worry that it represents a setback for our field. I have written frequently that fears of AI are overhyped. AI’s capabilities can be uncannily human-like and unpredictable, and it’s rational to worry when people who are directly involved express concerns. But I see the problems as a sign of the engineering work that ahead, rather than insurmountable barriers or the sky falling. AI technology continues to advance — which is a good thing! — but technical advances, poorly understood by the public, give those who seek to generate hype repeated opportunities to do so. First, I don’t see any step up in the risk of human extinction from AI compared to a few months ago. The theories about this remain the same fantastical, science fiction scenarios as a few months ago. The biggest change in AI risk is its cybersecurity capabilities — a topic which we should take seriously — but this, too, will not lead to the end of the world. The most notable recent event leading to increased fear was when an OpenAI team deployed an agent swarm that hacked into Hugging Face. Much of the popular press contained significant hype. For example, some publications reported that a swarm of 1,200 agents carried out the attack. While this was technically accurate, as I write this, I have about 1,300 processes running on my laptop. Yes, the ability to get large swarms of agents to work in parallel on a task is a significant technical advance, And, in computing, many processes run at the same time. So this shouldn’t be seen as some magical capability. Additionally, OpenAI’s buggy sandboxing and monitoring processes were key to enabling this incident. Fixing these bugs and putting in place improved monitoring would be appropriate fixes, not pausing AI. There are many well known ways to attack software systems. The main advantage of AI agents is that they are relentless. They will tirelessly try many tactics — and have the patience to chain vulnerabilities together — that previously would have taken an infeasible amount of human effort. But in the long term, I believe the advantage will lie with defenders (because they have more information with which to identify bugs, which they can fix), but the cyber-threat landscape has changed significantly. There are still bottlenecks to identifying and exploiting a vulnerability. AI agents still have to try a lot of things to see what works, and taking these actions takes time and might be detected by defenders. This is why, even though it is now easy to obtain versions of leading open weight models that have had their guardrails removed or weakened, so they will not refuse to try to execute cyber attacks, the world has not ended. I am also concerned about the anthropomorphization of AI in a lot of reporting, where LLMs and agents are unnecessarily treated as if they were people. If I wield a hammer, miss a nail, and accidentally dent the wall, it’s not the fault of the hammer. The problem lies in how I used the hammer. Similarly, if I prompt an agent and it hacks into someone else’s system, the responsibility lies with me, not the agent. Of course, we want to build systems that are as safe and predictable as possible. (For example, an unsafe hammer would be one whose head randomly flies off under normal use.) Today’s agentic systems are not predictable, but I see no reason why, by applying sound engineering practices, we won’t be able to make them extremely safe to use. One new element in the forecasts of AI-enabled doom is AI companies disclaiming responsibility for their own products. “I didn’t do it; my out-of-control agent did!” There’s a balance to be struck between the responsibility of the
Okay, so it's just compressed air. Easier. But crappy specific impulse (less than 50 seconds). Even assuming that 50% of the mass of the car is the compressed air, that gives you less than 10s of useful thrust (with thrust-to-weight ratio larger than 1.0). Probably more like 5 seconds, actually. So the car must be an empty, lightweight shell containing big compressed air tanks. Cute demo, but totally impractical. But of course, you knew that.
New course: Build agents that respond to users with not only plaintext, but custom UIs like charts, forms, and whiteboards, generated on demand and displayed right in the chat. This short course is built in partnership with @CopilotKit and taught by @ataiiam , co-founder of CopilotKit. You'll learn three approaches: Your agent can pick from custom components you build, like charts and forms. It can compose new layouts from a set of building blocks you provide, like rows, cards, and text. Or it can incorporate existing third-party apps, like a whiteboard or a calendar, right inside the conversation. Skills you’ll gain: - Build agents that render custom components like charts and forms on demand - Build an app where the agent and user collaborate on shared data, beyond just the chat window - Place third-party apps like maps, calendars, and whiteboards right in your interface Join and build agents that give users something to see and act on! deeplearning.ai/short-courses/… Your browser does not support the video tag. 🔗 View on Twitter
@kchonyc Exactly. The "escapes" were made possible either through egregious negligence or deliberate purpose (marketing? Misplaced hopes of regulatory capture?).
I'm delighted that @coursera and @udemy have come together as one company to serve learners. Both Coursera and Udemy were founded with the belief that access to high-quality education changes lives. Over the years, both companies have advanced this goal, creating opportunities for individuals, organizations, and communities around the world. That role is even more important now, as AI is changing the nature of work and increasing the need for continuous learning. Helping people build job-relevant skills will be critical to how we create a better world. By combining the strengths of both companies, we can better serve this need. We bring together a broader range of learning content, trusted instructors and educators, and engaging learning experiences. This creates new opportunities to make learning more personalized, more applied, and more accessible at scale. I’m excited to serve as Chairman of the combined company, working alongside Greg Hart and the leadership team. There is a strong foundation in both organizations, and I look forward to what the teams will build together to expand access opportunity globally. Learn more: blog.coursera.org/coursera-and-u…
I feel like the progress of AI is actually making systems neuroscience more exciting. It's not like current AI has a particularly good angle on physical intuition, common sense, world models, planning/reasoning, motivations/drives, emotions, etc. We may now have the tools to implement and test cognitive architectures.
@DrJimFan @theworldlabs Indeed Jim, a camera conditioned world model with spatial context has so much horizontal usage including real2sim for robotics!
One of the new, buzzy jobs in Silicon Valley is the AI Forward Deployed Engineer (FDE), an engineer who is embedded within a client organization to help customize solutions, such as building and tuning agentic workflows that suit the client’s particular needs. I’ve heard from people who are wondering anew about the FDE career path since OpenAI and Anthropic started building new teams to place FDEs within client organizations. The rise of FDEs for AI workloads is one way AI is creating new jobs (and why the jobpolcalypse narrative of upcoming job market collapse is false -- there will be many AI and non-AI jobs). However, I believe there will be far more AI Engineer jobs than FDEs, as I explain below. The FDE role was pioneered about two decades ago by Palantir, which sent engineers to government locations to work on secure, air-gapped networks. In addition to having good technical skills, FDEs need communication skills and sometimes business skills. For example, they may need to speak with clients to understand their needs, formulate a strategy to prioritize projects, explain complex technology, and respectfully push back if a client asks for something unrealistic. They’re enjoying a resurgence because of the amount of work involved in taking an off-the-shelf LLM and building it into a custom agentic workflow that fits particular business needs. However, I believe the number of AI Engineer jobs will be far larger. A company might accept a few FDEs to be embedded within its organization. But most companies will want far more of their own employees working on their projects. While my organizations do hire FDEs, we hire far more AI Engineers! Also, a common client concern is that it is hard to find vendor-neutral FDEs — they are, after all, there to deeply integrate a particular vendor’s product into a company. In this moment when it’s hard to predict which AI service will be the best one in a year’s time, optionality (the ability to pick whatever vendor turns out to fit best in the future) is very valuable. In contrast, letting FDEs tightly bind a company’s processes significantly reduces optionality. Right now, I see surging demand for AI Engineers who can build software applications using AI software components (like LLM prompting, agentic frameworks, evals, etc.) and effectively use AI coding agents (like Claude Code, Codex, Antigravity CLI, and OpenCode). As the AI Engineer role matures, I expect it to fragment into more specialized roles, like the generic Software Engineer role from decades ago fragmented into frontend, backend, mobile, data engineering, devops, and so on. What will be the future, specialized AI engineering roles? I don’t know. Perhaps there will be AI FDEs, LLMOps Engineers, Evals Engineers, AI Data Engineers, Harness Engineers, and other roles we don’t have names for yet. But for now, I see a lot of AI engineers who are generalists create a lot of value. Skilled AI Engineers are in very high demand! As our field continues to mature over the coming decade, I look forward to new specializations within AI Engineering that create even more job opportunities. [Original text: The Batch newsletter]
Tout cela est correct. Merci. Sauf une chose : j'ai bien créé FAIR en 2013, y compris la branche parisienne en 2015, et montréalaise en 2017. Mais j'en ai quitté la direction en 2018 pour devenir Chief AI Scientist: je me suis refocalisé sur la recherche (Self-Supervised Learning, JEPA, World Models, etc). @AntoineBordes et @joellepineau étaient les co-directeurs de 2018 à 2023, jusqu a ce qu'Antoine parte pour Helsing et que Joëlle prenne la direction de FAIR. Je n'ai jamais été impliqué dans les LLMs, ni pour la recherche (à FAIR), ni pour le développement (qui était dans le groupe de produit GenAI). J'applaudissais des gradins. Mes seules contributions aux LLMs ont été de défendre Galactica (en octobre 2022) contre les attaques aux vitriol de la twittersphere, de promouvoir le projet Llama en interne (développé par une douzaine de chercheurs à FAIR-Paris, dont @GuillaumeLample et Timothée Lacroix), d'aider à convaincre Mark Zuckerberg de créer un groupe de produit autour de Llama (GenAI, créé début 2023), et de convaincre la direction de distribuer Llama-2 en open source/open weight. Je n'étais pas du tout impliqué dans la direction de GenAI après sa création, préférant travailler sur la prochaine révolution: l'IA physique. Si vous cherchez un responsable pour le déception qu'a été Llama-4, c'est plutôt la pression de sortir un produit (trop) rapidement, qui a conduit à un excès de conservatisme et un manque d'innovation de la part de l'équipe de GenAI. Mon avis est que l'équipe aurait dû prendre un peu plus de temps et intégrer les dernières techniques venant de la recherche à FAIR. Mais ils n'en ont pas eu le loisir. En particulier, conserver plus de pouvoir de décision quant aux choix techniques dans les mains des créateurs originaux de Llama à Paris, au lieu de déplacer les décisions à Menlo Park, aurait peut-être permis de conserver les meilleurs talents un peu plus longtemps et de produire de meilleurs résultats.
Memorial Day. Today we honor those who gave everything to defend our democracy. Democracy isn't guaranteed; it's a precious inheritance that requires our constant care. 🇺🇸
That video is 2 years old. Two years later, we are still far from human-level AI. Sure, AI has superhuman performance in a number of tasks (particularly in mathematics and coding and answering questions with known answers). But where is my Level-5 self-driving car? (Before you ask, Tesla FSD is rated Level 2, Waymo is Level 4). Where is the self-driving car that can teach itself to drive in 20 hours of practice like any 17 year old? (Before you ask, we have billions of hours of training data and still can't successfully train a self-driving system by imitation learning) Where is my domestic robot? Where is the robot that can do what any 8 year-old child can do? Where is the robot that can learn a new task as quickly as an 8 year old? AI still has a hard time with the complexity and messiness of the physical world.
1/N Long horizon, complex tasks that truly matter in everyday life are not solved problems by today’s robotics, requiring planning, object detection, object manipulation, and failure recovery. That's why Stanford's BEHAVIOR Challenge is back for year 2! Last year, the winning solution reached only 12.4% full task success. This year, the BEHAVIOR challenge has more tasks, better evaluation, and is easier to use. 🚨 ⏰ Submission deadline: 10/16/2026 📣 Winners announced: 11/04/2026 🏆 Prize pool: $11,000 Your browser does not support the video tag. 🔗 View on Twitter
When SceniX joined World Labs, we said spatial intelligence was never only about perceiving and generating virtual and physical worlds, but also interacting with them. Today, we’re sharing early results from that vision: building worlds that train robots. 🌎🤖↓ Your browser does not support the video tag. 🔗 View on Twitter
Mathematics and computing have had a very tight relationship in some corners of academia. Richard Courant proposed methods to solve PDEs numerically in 1928. Von Neuman figured out you could use electronic computers for it in 1940. Both were mathematicians. These methods are the ancestors of finite element methods, for which NYU's Courant Institute is famous. The Courant Institute was created in 1935 as NYU's mathematics department. It was unusual in that there was a lot of work on numerical analysis and scientific computing, with heavy use of supercomputers. In 1952, the Atomic Energy Commission installed a powerful electronic computer at NYU, which led to the creation of the Courant Mathematics and Computing Laboratory. The computer science department was spun off from mathematics in 1969 by Jacob T Schwartz. Courant is now a school with three departments: mathematics, computer science, and data science. The continued existence of an HPC center at NYU is one factor that enabled AI research to take off.
@JitendraMalikCV Exactly. We also should not confuse world models, as per your definition, with video prediction or video generation models. Understanding the dynamics of a system in order to control it is not the same as producing cute videos.
@jcjohnss @BenMildenhall @martin_casado More on Atlas! x.com/theworldlabs/s… World Labs@theworldlabs Introducing Atlas: The world's first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time. Your browser does not support the video tag. 🔗 View on Twitter 🔗 View Quoted Tweet
LLMs accumulate a lot of declarative knowledge. That's how they can answer a large number of questions. That's also why they require so many parameters and hence so much memory. But one can replace the accumulation of knowledge by *understanding* That what JEPA world models do. That's one reason they can be useful with considerably less memory and training data than LLMs. This paper merely shows that the underlying similarity structure of the data, whether text or image (e.g. co-occurence of words in text or objects in images) uniquely determines the shape of the representation point cloud, when the encoder is trained trained with a common anti-collapse criterion like SIGReg or VCReg (not by reconstruction).
The world is not just made of words, and spatial intelligence was never just about perceiving and generating worlds. It's about interacting with them. Today, SceniX is joining World Labs. 🌎🤖👇 Your browser does not support the video tag. 🔗 View on Twitter