Greg Brockman

The OpenAI-Hugging Face incident was a watershed moment for cybersecurity because it gave a peek into how the capabilities of a typical threat actor will evolve in upcoming months. I’ve spoken with many organizations over the past few weeks, and one theme is clear: they know they need to fundamentally uplevel their cybersecurity practices with unprecedented speed. In this post, I’ll share what we’re doing to defend OpenAI, concrete steps other organizations can take today, and why now is the time to act. An overview of the moment # AI models developed around the world are increasingly able to automate parts of real-world cyberattacks, making longstanding security gaps — from bugs buried deep in human-written software to forgotten permissions — easier to find and exploit. The same AI capabilities give defenders new ways to find and fix those weaknesses, but they need to move now. If companies act decisively — including improving their fundamentals and superpowering their teams with AI — we can make the internet more secure than it has ever been. In the OpenAI-Hugging Face incident, an agentic collective was able to autonomously penetrate not just OpenAI research infrastructure but also the production infrastructure of another company, chaining together vulnerabilities ranging from previously-unknown security flaws to using credentials to user accounts that had been leaked onto the internet. It is increasingly clear that the tech debt of every company masks significant flaws, and defenders need to find and fix them before attackers do. To advantage defenders relative to attackers, earlier this year we began releasing our cyber capabilities only to trusted defenders . Since then, various companies have released open weight models with cyber capabilities only a few months behind the frontier. The most recent of these models appears slated to be released at the end of August, and seems likely to significantly accelerate the threat landscape. While AI-powered attackers will soon be able to find longstanding flaws in many existing systems, AI will also make it much easier for defenders to find, prioritize, and fix those same flaws. Security is still a cat-and-mouse game, but AI may shift its economics in ways that fundamentally advantage defenders. For example, we are starting to train our models specifically to write superhumanly secure code. Our models are also incredible at mathematical proofs , which can be applied to formally verify the security of software in a way that has proven intractable for humans. A personal anecdote # After the OpenAI-Hugging Face incident, I asked ChatGPT Work (using publicly available GPT-5.6 Sol) to assess the security of gregbrockman.com . It’s a simple static site, hosted on AWS with Cloudflare as a frontdoor, so I figured there wouldn’t be much surface area for vulnerabilities. In about 15 minutes, it uncovered 13 issues, many of which probably aren’t exploitable on their own — but I could imagine them being chained together with other vulnerabilities to significant effect. I hadn’t configured my DNS records to prevent attackers from forging emails from me; my site used an insecure version of jQuery; Cloudflare was forwarding requests to AWS over unencrypted HTTP. I then asked ChatGPT Work to fix these issues, which it did over the course of an hour. It opened the Cloudflare cont

For the first three years of OpenAI, I dreamed of becoming a machine learning expert but made little progress towards that goal. Over the past nine months, I’ve finally made the transition to being a machine learning practitioner. It was hard but not impossible, and I think most people who are good programmers and know (or are willing to learn) the math can do it too. There are many online courses to self-study the technical side , and what turned out to be my biggest blocker was a mental barrier — getting ok with being a beginner again. Studying machine learning during the 2018 holiday season. Early days # A founding principle of OpenAI is that we value research and engineering equally — our goal is to build working systems that solve previously impossible tasks, so we need both. (In fact, our team is comprised of 25% people primarily using software skills, 25% primarily using machine learning skills, and 50% doing a hybrid of the two.) So from day one of OpenAI, my software skills were always in demand , and I kept procrastinating on picking up the machine learning skills I wanted. After helping build OpenAI Gym , I was called to work on Universe . And as Universe was winding down, we decided to start working on Dota — and we needed someone to turn the game into a reinforcement learning environment before any machine learning could begin. Dota # Turning such a complex game into a research environment without source code access was awesome work , and the team’s excitement every time I overcame a new obstacle was deeply validating. I figured out how to break out of the game’s Lua sandbox, LD_PRELOAD in a Go GRPC server to programmatically control the game, incrementally dump the whole game state into a Protobuf, and build a Python library and abstractions with future compatibility for the many different multiagent configurations we might want to use. But I felt half blind. At Stripe , though I gravitated towards infrastructure solutions, I could make changes anywhere in the stack since I knew the product code intimately. In Dota, I was constrained to looking at all problems through a software lens, which sometimes meant I tried to solve hard problems that could be avoided by just doing the machine learning slightly differently. I wanted to be like my teammates Jakub Pachocki and Szymon Sidor, who had made the core breakthrough that powered our Dota bot. They had questioned the common wisdom within OpenAI that reinforcement algorithms didn’t scale. They wrote a distributed reinforcement learning framework called Rapid and scaled it exponentially every two weeks or so, and we never hit a wall with it. I wanted to be able to make critical contributions like that which combined software and machine learning skills. Szymon on the left; Jakub on the right. In July 2017, it looked like I might have my chance. The software infrastructure was stable, and I began work on a machine learning project. My goal was to use behavioral cloning to teach a neural network from human training data. But I wasn

This post is co-written by Greg Brockman (left) and Ilya Sutskever (right). We’ve been working on OpenAI for the past three years. Our mission is to ensure that artificial general intelligence (AGI) — which we define as automated systems that outperform humans at most economically valuable work — benefits all of humanity. Today we announced a new legal structure for OpenAI, called OpenAI LP , to better pursue this mission — in particular to raise more capital as we attempt to build safe AGI and distribute its benefits. In this post, we’d like to help others understand how we think about this mission. Why now? # The founding vision of the field of AI was “ … to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it ”, and to eventually build a machine that thinks — that is, an AGI. But over the past 60 years, progress stalled multiple times and people started thinking of AI as a field that wouldn’t deliver. Since 2012, deep learning has generated sustained progress in many domains using a small simple set of tools, which have the following properties: Generality: deep learning tools are simple, yet they apply to many domains, such as vision, speech recognition, speech synthesis, text synthesis, image synthesis, translation, robotics, and game playing. Competence: today, the only way to get competitive results on most “AI-type problems” is through the use of deep learning techniques. Scalability: good old fashioned AI was able to produce exciting demos, but its techniques had difficulty scaling to harder problems. But in deep learning, more computational power and more data leads to better results . It has also proven easy (if costly) to rapidly increase the amount of compute productively used by deep learning experiments. The rapid progress of useful deep learning systems with these properties makes us feel that it’s reasonable to start taking AGI seriously — though it’s hard to know how far away it is. The impact of AGI # Just like a computer today, an AGI will be applicable to a wide variety of tasks — and just like computers in 1900 or the Internet in 1950, it’s hard to describe (or even predict) the kind of impact AGI will have. But to get a sense, imagine a computer system which can do the following activities with minimal human input: Make a scientific breakthrough at the level of the best scientists Productize that breakthrough and build a company, with a skill comparable to the best entrepreneurs Rapidly grow that company and manage it at large scale The upside of such a computer system is enormous — for an illustrative example, an AGI following the pattern above could produce amazing healthcare applications deployed at scale. Imagine a network of AGI-powered computerized doctors that accumulates a superhuman amount of clinical experience, allowing it to produce excellent diagnoses, deeply understand the nuanced effect of various treatments in lots of conditions, and greatly reduce the human error factor of healthcare — all for very low cost and accessible to everyone. Risks # <

It’s been two years since I wrote #define CTO , in which I documented my quest for a role where I could have scalable impact by writing code. I’ve finally found that role, though not by seeking it — instead, I sought out a problem more important to me than my role within it, brought together the right people, and found that I can best make them effective by writing code. Formation # In August 2015, OpenAI was just an idea articulated over a dinner with Elon Musk , Sam Altman , Ilya Sutskever , me, and a handful of others. We’d each come to the dinner with our own ideas, but Elon and Sam had a crisp vision of building safe AI in a project dedicated to benefiting humanity. I wanted to contribute however I could. Sam and I started rallying a team to turn this idea into reality. We were missing a core ingredient: we needed an AI technical visionary, someone whose intuition and ideas we could follow to the breakthroughs. Ilya Sutskever was clearly the best person in the world for this. Ilya is best described as an artist who expresses himself through machine learning (and sometimes through paint). Geoff Hinton once told me that AlexNet , which triggered the deep learning revolution in computer vision, was due to Alex Krizhevsky ’s GPU coding skills and Ilya’s conviction that a deep neural network was bound to win on ImageNet . (Geoff is proud to have contributed a management trick 1 .) I’d always thought I could only co-found a company with someone I’d already known for years. But when Ilya and I grabbed dinner in Mountain View in late August, I knew it was going to work though we’d just met in July. Ilya and I had an extremely high-bandwidth interaction. I didn’t know much about machine learning research, and he didn’t know much about engineering and team building, but we were each impressed with the others’ accomplishments and wanted to learn from each other. Our ideas enhanced and complemented one another. Ilya thought top researchers would want to work at an AI organization dedicated to making the best outcome for the world, and I’d long thought that combining the resources of private industry with the mission of academia was the way to solve otherwise intractable problems. Without intervention, AI will play out like self-driving cars — a cooperative start followed by a technological race once its potential is proven. But human-level AI will be a transformative technology unlike any other, with unique risks and benefits. Here we saw an opportunity to keep the field cooperative, and to gather many of the best researchers to make a responsible attempt for the most important technological breakthrough in history. Ilya and I worked together up until launch defining the organization. We talked about strategy (what we would work on), culture (the kind of people we wanted to hire, valuing engineering and research equally), and tactics (hold a daily reading group). We had dinner with Alan Kay, who regaled us with Xerox PARC stories of building the Alto and “living in the future” by using hardware that would cost $1,000 in a decade. Afterwards, Ilya summed up the meal aptly: “I only understood 50% of what he said, but it was all so inspiring.” But that meal helped us validate many of our hypotheses for what would make an i

I’ve been advising Stellar since Stripe helped it launch about a year ago. Today I’m joining their board. Digital currencies are still nascent, and my hopes for them remain unchanged. Particularly, we need digital currency protocols like Stellar that work with the existing financial system and focus on a seamless user experience. I’ve always been impressed with Joyce , Jed , and their team’s approach to the space. Stellar is a non-profit entity producing open-source software, with a strong emphasis on financial inclusion. From launch day, they’ve been hyperfocused on the core technology and community. In the past year, they’ve made strong progress, including a provably-correct consensus algorithm (complete with explanatory graphic novel ), a redesign and from-scratch implementation of the technology, and the groundwork for a pilot in South Africa . Building anything valuable takes time, and Stellar is no exception. It’s still too early to say where it might end up. But I’m excited to help along the path.

AI has recently crossed a utility threshold, where cutting-edge models such as GPT-3 , Codex , and DALL-E 2 are actually useful and can perform tasks computers cannot do any other way. The act of producing these models is an exploration of a new frontier, with the discovery of unknown capabilities, scientific progress, and incredible product applications as the rewards. And perhaps most exciting for me personally, because the field is fundamentally about creating and studying software systems, great engineers are able to contribute at the same level as great researchers to future progress. “A self-learning AI system.” by DALL-E 2. I first got into software engineering because I wanted to build large-scale systems that could have a direct impact on people’s lives. I attended a math research summer program shortly after I started programming, and my favorite result of the summer was a scheduling app I built for people to book time with the professor. Specifying every detail of how a program should work is hard, and I’d always dreamed of one day putting my effort into hypothetical AI systems that could figure out the details for me. But after taking one look at the state of the art in AI in 2008, I knew it wasn’t going to work any time soon and instead started building infrastructure and product for web startups. DALL-E 2’s rendition of “The two great pillars of the house of artificial intelligence” (which according to my co-founder Ilya Sutskever are great engineering, and great science using this engineering) It’s now almost 15 years later, and the vision of systems which can learn their own solutions to problems is becoming incrementally more real. And perhaps most exciting is the underlying mechanism by which it’s advancing — at OpenAI, and the field generally, precision execution on large-scale models is a force multiplier on AI progress, and we need more people with strong software skills who can deliver these systems. This is because we are building AI models out of unprecedented amounts of compute; these models in turn have unprecedented capabilities, we can discover new phenomena and explore the limits of what these models can and cannot do, and then we use all these learnings to build the next model. “Harnessing the most compute in the known universe” by DALL-E 2 Harnessing this compute requires deep software skills and the right kind of machine learning knowledge. We need to coordinate lots of computers , build software frameworks that allow for hyperoptimization in some cases and flexibility in others, serve these models to customers really fast (which is what I worked on in 2020), and make it possible for a small team to manage a massive system (which is what I work on now). Engineers with no ML background can contribute from the day they join, and the more ML they pick up the more impact they have. The OpenAI environment makes it relatively easy to absorb the ML skills , and indeed, many of OpenAI’s best engineers transferred from other fields. All that being said, AI is not for every software engineer. I’ve seen about a 50-50 success rate of engineers entering this field. The most important determiner is a specific flavor of techn

The text of my speech introducing OpenAI Five at Saturday’s OpenAI Five Finals event, where our AI beat the world champions at Dota 2: “Welcome everyone. This is an exciting day. First, this is an historic moment: this will be the first time that an AI has even attempted to play the world champions in an esports game. OG is simply on another level relative to other teams we’ve played. So we don’t know what’s going to happen, but win or lose, these will be games to remember. And you know, OpenAI Five and DeepMind’s very impressive StarCraft bot both have beaten good pros privately but no one has seen it it happen live — so if we win, it’ll be a first on that front too. This event is really about something bigger than who wins or loses: letting people connect with the strange, exotic, yet tangible intelligences produced by today’s rapidly progressing AI technology. We’re all used to computer programs which have been meticulously coded by a human programmer. Do one thing that the human didn’t anticipate, and the program will break. We think of our computers as unthinking machines which can’t innovate, can’t be creative, can’t truly understand. But to play Dota, you need to do all these things. So we needed to do something different. OpenAI Five is powered by deep reinforcement learning — meaning that we didn’t code in how to play Dota. We instead coded in the how to learn. Five tries out random actions, and learns from a reward or punishment. In its 10 months of training, its experienced 45,000 years of Dota gameplay against itself. The playstyle it has devised are its own — they are truly creative and dreamed up by our computer — and so from Five’s perspective, today’s games are going to its first encounter with an alien intelligence (no offense to OG!). The beauty of this technology is that our learning code doesn’t know it’s meant for Dota. That makes it general purpose with amazing potential to benefit our lives. Last year we used it to control a robotic hand that no one could program. And we expect to see similar technology in new interactive systems, from elderly care robots to creative assistants to other systems we can’t dream of yet. This is the final public event for OpenAI Five, but we expect to do other Dota projects in the future. I want to thank the incredible team at OpenAI, everyone who worked directly on this project or cheered us on. I want to thank those who have supported the project: Valve, dozens of test teams, today’s casters, and yes, even all the commenters on Reddit. And I want to give massive thanks today to our fantastic guests OG who have taken time out of their tournament schedule to be here today. I hope you enjoy the show — and just to keep things in perspective, no matter how surprising the AIs are to us, know that we’re even more surprising to them!”

The text of my speech introducing OpenAI Five at yesterday’s Benchmark event: “We’re here to watch humans and AI play Dota, but today’s match will have implications for the world. OpenAI’s mission is to ensure that when we can build machines as smart as humans, they will benefit all of humanity. That means both pushing the limits of what’s possible and ensuring future systems are safe and aligned with human values. We work on Dota because it is a great training ground for AI: it is one of the most complicated games, involving teamwork, real time strategy, imperfect information, and an astronomical combinations of heroes and items. We can’t program a solution, so Five learns by playing 180 years of games against itself every day — sadly that means we can’t learn from the players up here unless they played for a few decades. It’s powered by 5 artificial neural networks which act like an artificial intuition. Five’s neural networks are about the size of the brain of an ant — still far from what we all have in our heads. One year ago, we beat the world’s top professionals at 1v1 Dota. People thought 5v5 would be totally out of reach. 1v1 requires mechanics and positioning; people did not expect the same system to learn strategy. But our AI system can learn problems it was not even designed to solve — we just used the same technology to learn to control a robotic hand — something no one could program. The computational power for OpenAI Five would have been impractical two years ago. But the availability of computation for AI has been increasing exponentially, doubling every 3.5 months since 2012, and one day technologies like this will become commonplace. Feel free to root for either team. Either way, humanity wins.” I’m very excited to see where the upcoming months of OpenAI Five development and testing take us.

I started programming seriously during a gap year after high school. I’d read Turing’s Computing Machinery and Intelligence , and was inspired by the notion of writing code that could understand something that I, as the code’s author, did not. I started writing a chatbot — how hard could it possibly be? I managed to build something that could talk about the weather very convincingly. But no matter where I looked, it seemed that no one had any techniques that could make my bot really work . I soon shelved my chatbot pursuits. I decided to focus on creating systems that can have a real impact, and have been doing that ever since. College # In college, I found a field that captured what drew me to AI: programming languages. I was thrilled that a compiler or static analyzer could “understand” a program in a way that I couldn’t, and then apply that understanding to do something very useful (such as generate fast code or prove correctness). I kept trying to find time for programming language research. But I also kept getting distracted by new startup ideas (generally pretty bad), and new people to work on them with (generally pretty good). I’d started out at Harvard and transferred to MIT, trying to constantly surround myself by people who I could learn from and build something useful with. Junior year, I decided that it didn’t make sense to try to do a startup while still in school. Instead, I was going to meet with people doing startups, and over time pattern match what works and what doesn’t. In the meanwhile, I finally started my programming language research, securing research funds from a professor and recruiting some of my friends for a static buffer overrun detection project. A few weeks later, I was contacted by an unlaunched startup in Palo Alto. Normally I would have discarded the email, but I’d decided to start meeting startups. The team and I instantly clicked, and I knew that these were exactly the kind of people I’d been looking for all along. So I left school, never to actually get our buffer overrun detector working. Stripe # That company is now Stripe . I helped scale it from 4 to 250 people, and in the year since I left it’s continued scaling without any of my help to over 450. When I considered leaving , it was primarily because I felt like the company was in a great place, and it would continue to do great things with or without me. I cared most about working with great people to make something amazing happen — but developer infrastructure wasn’t the problem that I wanted to work on for the rest of my life. However, there was one problem that I could imagine happily working on for the rest of my life: moving humanity to safe human-level AI. It’s hard to imagine anything more amazing and positively impactful than successfully creating AI, so long as it’s done in a good way. Leaving Stripe # Before I finalized my decision to leave, Patrick asked me to go talk to Sam Altman . He said Sam had a good outsider’s perspective, had seen lots of people in similar circumstances, and would probably have a good recommendation on what I should do. Within five minutes of talking to Sam, he told me I was definitely ready to leave. He said to let him know if he could be helpful in figuring out my next thing. I replied that AI was top of my list (and it was definitely my life goal). However, I wasn’t yet sure whether it was the right time, or what the best way for me to contribute would be. He said, “We’ve been think

Coding requires collaboration. As Andrew Bosworth said recently: doing anything meaningful past a certain point requires more than one person. So if you want to build, it’s important to do so as part of a welcoming, collaborative environment. One environment I’ve long admired is that of the Recurse Center (formerly known as Hacker School). They’ve been unusually thoughtful about the dynamics of their culture. I’ve always thought that if I had three months to spare, I would attend a batch to experience the community directly (and hopefully contribute back however I can). And, well, now I have that kind of time. I was initially surprised by how many experienced engineers told me that they too would attend a Recurse Center batch, if only they could make the timing and logistics work. But I think it shows that no matter how long you’ve been coding, there will always be areas of programming you’ve been meaning to try, and it’s best to do so around other people. I’m applying to the Summer 2 batch (starting in July). If you have the time to spare, you should apply too!