Newsletter #24: Why was Sam Altman fired? Did Q* and early signs of Artificial General Intelligence contribute?

When Sam Altman was fired by OpenAI on November 17th, it felt like one of those events where you might not forget where you were when you first heard about it.

I know where I was…I was sitting with my wife Julia La Roche in our living room and she saw it on X/Twitter first…

Julia: “Sam Altman just got fired from OpenAI!?!?!”

Me: “Yeah right.”

Julia: “No, I’m serious.”

Me: “Impossible…wait, really? Why would they fire him?!?!?! That’s insane…”

While the question remains unanswered other than a vague comment from the Board of OpenAI, a lot has happened in the last 10 days.

“Mr. Altman’s departure follows a deliberative review process by the board, which concluded that he was not consistently candid in his communications with the board, hindering its ability to exercise its responsibilities. The board no longer has confidence in his ability to continue leading OpenAI” (link).

A clear explanation beyond the above wasn’t published and Greg Brockman Co-Founder and President quit soon after he learned of the Board’s decision to fire Sam.

The next day, OpenAI COO explained to employees that the firing of Sam ‘‘took [the management team] by surprise’ and that management had had ‘multiple conversations with the board to try to better understand the reasons and process behind their decision…”

“We can say definitively that the board’s decision was not made in response to malfeasance or anything related to our financial, business, safety, or security/privacy practices…This was a breakdown in communication between Sam and the board…”

“We still share your concerns about how the process has been handled, are working to resolve the situation, and will provide updates as we’re able.” (“No ‘malfeasance’ behind Sam Altman’s firing, OpenAI memo says” by Axios 11.18.23)

OpenAI’s board initially refused to bring Sam back but after “OpenAI was in open revolt on Monday with more than 730 employees signing an open letter threatening to leave unless the board resigns and reinstates Sam Altman as CEO, along with co-founder and former president Greg Brockman.” (“OpenAI Staff Threaten to Quit Unless Board Resigns” via WIRED 11.20.23)

On November 20th, Ilya Sutskever, Co-Founder and Chief Scientist, reversed course on initially voting in favor of firing Sam…

Culminating with the 11.22 announcement that Sam and OpenAI reached an agreement for him to return as CEO along with an “initial’ new board”...

The thing that hasn’t changed over the last 10 days is that we still don’t really know why Sam Altman was fired.

Recent reports have surfaced around the role OpenAI’s “Q* model” and associated Artificial General Intelligence (AGI) breakthroughs could have played in Sam’s firing.

While it’s still just speculation at this point, the idea that signs of AGI were surfacing ahead of Sam’s firing which exacerbated the tension between OpenAI’s overarching non-profit mission and its capped for-profit subsidiary seem like a reasonable explanation of how these events might have developed.

Whereas LLMs to date effectively have intuition associated with the massive knowledge bases they’re trained on aka “a Zip file of a chunk of the Internet,” the Q* model might have shown signs of the ability to think, logically reason and develop new ideas.

Aka the beginning of turning the corner from Systems 1 thinking to Systems 2 thinking (more on that below).

“Some at OpenAI believe Q* (pronounced Q-Star) could be a breakthrough in the startup's search for what's known as artificial general intelligence (AGI), one of the people told Reuters. OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks.”

“Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because the individual was not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said.”

“Researchers consider math to be a frontier of generative AI development. Currently, generative AI is good at writing and language translation by statistically predicting the next word, and answers to the same question can vary widely. But conquering the ability to do math — where there is only one right answer — implies AI would have greater reasoning capabilities resembling human intelligence.”

“This could be applied to novel scientific research, for instance, AI researchers believe.”

“Unlike a calculator that can solve a limited number of operations, AGI can generalize, learn and comprehend.” (“OpenAI researchers warned board of AI breakthrough ahead of CEO ouster, sources say” by Reuters 11.23.23)

This explanation of Sam’s firing seems feasible given the challenges OpenAI has had to navigate to maintain alignment with its non-profit mission while pursuing commercialization of its products via its “new capped profit arm” (OpenAI: “Our Structure”) to fuel massive investments in product development.

Taking a step back, it might be safe to say Artificial General Intelligence isn’t an “if” but a “when” and “how” which is where Andrej Karpathy’s recent video “The Busy Person’s Intro to LLMs” comes into play.

*Big s / o to my colleague Hugo Rios-Neto who helped bring Andrej’s video to my attention and hear more about his thoughts on what an LLM enabled Sports OS could look like ;)

Whereas we understand Operating Systems today to be Microsoft Windows or MacOS, Andrej brings to life how they might evolve, the role of AI and what the Large Language Model enabled Operating System 2.0 of the future is going to be and it’s absolutely fascinating.

While I hope anyone that reads this newsletter might give the video a shot from beginning to end, jump to 42 minutes and 16 seconds where Andrej “ties everything together into a single diagram” via the “Large Language Model Operating System.”

If / when that captures your attention, go back to the beginning and watch the whole thing. You won’t regret it.

By way of background, we use the operating systems on our phones, laptops etc today as the foundation for device functionality as they provide a stable platform for a wide range of software applications we use daily including communication, productivity, games and utilities.

Now when you start thinking about an LLM enabled Operating System, one that embeds intuitive thinking along with reasoning into the OS so that it gets to know you, the way you work and live as well as how to take care of increasingly complex tasks on your behalf etc, you can see why there is so much excitement around developments such as the Q* project.

The interface becomes exponentially more intuitive due to the natural language interface while excelling at understanding and contextualizing increasingly complex user requests, similar to the way a rockstar Chief of Staff operates.

Continuously learning, improving its performance based on user interactions, feedback, behavioral data and done so with a photographic memory.

And yes, now might be a good idea to watch Her (link) starring my man Joaquin Phoenix if you haven’t already. ;)

Given the context above, the last 9 days of OpenAI developments and all of the conversations developing around AGI’s potential arrival, there are 2 ideas central to Andrej’s video that I wanted to highlight.

1: System 1 vs System 2 Thinking

Today’s LLMs are effectively “gestalts” of a Zip file of a big chunk of the Internet. They draw all sorts of connections between the content they analyze which effectively enables them to provide quick answers to questions by making statistical predictions about what “the right answer” is to your question based on the content they’ve analyzed which at times can also lead to hallucinations as they effectively make the wrong connections. Similar to the way we use intuition to answer questions, especially about subjects we have a deep level of expertise and experience with. This is consistent with System 1 thinking.

System 2 thinking is when systems can think, reason and come up with new ideas. This is the type of thinking where more time typically increases the quality and quality of the results whereas more time doesn’t impact the results of System 1 thinking. This is where we step into AGI as more time does translate to better answers, just as it would if you gave a human being more time to solve a difficult question.

A simple but powerful example of System 1 vs System 2 thinking was brought to life by Andrej in the video at the 12 minutes and 30 seconds mark where he described something called the “Reversal Curse.”

Andrej explained that when you ask an LLM (i.e. GPT-4) “Who is Tom Cruise’s mother?” and it correctly answers “Mary Lee Pfeifer”, you would assume the model would then know the answer to the question “Who is Mary Lee Pfeiffer’s son?”

But instead of saying, “Tom Cruise”, it answers “I don’t know.”

System 1 thinking isn’t capable of connecting the dots whereas System 2 thinking in the future would.

2: LLMs Aren’t Chatbots, They’re the Kernel Process of an Emerging Operating System

“We can borrow a lot of analogies from the previous computing stack to try and think about this new computing stack fundamentally based around LLMs orchestrating tools for problem solving and accessible via natural language interface of language.” (45 mins 25 secs)

The emergence of AGI and the way the Operating System of the future could evolve accordingly is something else…

That’s it for this week.

Hope you enjoyed it and talk soon.

Alec