Newsletter #39: From Mad Libs to Alien Intelligence to Iron Man Suits: Rick Rubin, Jack Clark + Andrej Karpathy Decode AI Software

One of the most remarkable things about enterprise AI is the impact it’s having on software.

Whereas traditional software (SaaS) is a tool that’s built for humans to use, AI enabled software “has opinions about what it should do with itself” (Jack Clark co-Founder of Anthropic) due to the intelligence baked into it.

Whereas traditional software can be thought of as a wrench in a garage, waiting for a human to take it off the shelf to tighten a bolt, AI enabled software can be thought of as a system infused with an intelligence that enables it to operate with different degrees of autonomy and collaboration with humans.

AI enabled software is an “everything tool” but “it’s hard to picture how that changes things, having an everything tool” (Rick Rubin aka THE GOAT) because of how powerful and durable it is.

The implications on those that build it and the enterprises that rely on it can’t be overstated.

The focus of Newsletter #39 is to make the above real by mashing up excerpts from the outstanding interview of Jack Clark by Rick Rubin on his podcast "Tetragrammaton" with Andrej Karpathy's “Software in the AI era” presentation at Y Combinator.

Goes without saying but I hope anyone that reads this finds the time to watch both.

Outstanding.

Let’s do this.

Part 1: Rick Rubin’s interview with Jack Clark provides the ideal context before checking out Andrej Karpathy’s presentation.

Five of my favorite key takeaways from the phenomenal interview?

1️⃣ Mad Libs

You remember Mad Libs? I do too and I LOVED that game.

Best explanation of “How do you build an LLM?” goes to Jack Clark.

I’m obviously a bit biased given that he hit Rick with the Mad Libs reference…

Rick: “How do you create a large language model?”

Jack: “You take a whole load of data and then you play Mad Libs on that data. So if I say to you - Mad Lib is like fill in the blank, the word's missing - if I say to you "it's Christmas and someone's coming down the chimney, their name is..." Santa Claus, right? Your brain has filled it in.”

“So imagine that you're doing that but at the scale of not just sentences but paragraphs, tens of pages, hundreds of pages in different domains - science, math, literature. You gather this very large data set and then you are training it where you're knocking out chunks of it and you're training it to complete that. The idea being if I can make very complex predictions here I must have understood something really subtle and important about the underlying data set.”

“Through forcing it to make these predictions at such a large scale you force it to compress its own representation so that it develops shorthand.”

“Just like creativity - you're giving it an artificial constraint that forces it to come up with creative ways to represent that data to itself in the form of shorthand. But shorthand ends up being bound up in just how thinking works. You develop all kinds of shorthand ways of thinking through things and this is the same.”

“I think of it as a bottled up distillation of the data it's read but it's not the data.”

“It's a thing that has thought about that data and now when you boot it up and ask questions about it, it's re-representing some of that to itself using its shorthand, thinking about things.”

“That's partly why it has the capacity for creativity because it's not just saying "oh let me go and refer back to this library" - it doesn't have access to the library. It has to do it all by itself.”

2️⃣ A different kind of intelligence, think of it as an “alien intelligence”...let’s start with the “phone number test”

The short-term memory of an AI system that is profoundly different from that of a human being. It’s easy to overlook the importance of short-term memory but keep in mind, prompt engineering or even better, context engineering (tweet from Tobi Lutke) which is a tactical unlock that speaks to the opportunity to tap into the “alien intelligence” most effectively…

Rick: “How is it not like the human brain beyond dreaming in the way it works?”

Jack: “I think the phone number test is good here.”

“You and I, we can remember on a good day a phone number - maybe someone tells you to remember it, you can walk down the street, say it to yourself a few times.”

“Your short-term memory, everyone on the planet's short-term memory allows them to remember something on the order of five to 10 things at once."

"There are some outliers that can do more but I'd challenge you to find a person that could probably do that for more than 20 distinct things and I think the limit is more like 12 or 13 or so.”

“An AI system can hold in its head tens of thousands to hundreds of thousands of things simultaneously.”

“That's not just memory, it's your short-term memory accessible.”

“When you and I think we're not manipulating that many concepts at once, we're actually manipulating stuff that's bound by this short-term scratch pad.”

“The AI system doesn't have that same constraint and so when it's thinking it's thinking in a way wholly unlike people - it's reasoning using much more data concurrently than people ever do.”

“This stuff thinks with a huge amount of input data that it holds in its head all at once.”

3️⃣ Taking “alien intelligence” a step further, imagine having unlimited access to a helpful colleague that’s read every book in existence…

To take this one step further, what if the “alien intelligence” has read every book across each function? Each specific workflow? You get the idea…

Rick: “So do you think it's as much of a psychological tool in that respect?”

Jack: “It's actually almost equivalent to just having a helpful colleague."

"It's like having access to a helpful colleague that's read every book in existence and if you're having trouble you say 'I'm stuck on something' and they'll say 'Have you considered...?'

Rick: “So it's different than the same maniac going to the library and getting the anarchist cookbook?”

Jack: “The books - it's more like they go to the library and there's someone who works at the library who's read every single book in it and says 'Oh I see what you're trying to do, you need these eight books that have a lot of the knowledge and I've read them, let me explain it to you.'"

“This stuff thinks with a huge amount of input data that it holds in its head all at once. So it might be more analogous to a mirror or a pool that thinks - your reflections in it and there's some very strange cognition or complexity underlying it.”

“We have to come up with new language for this and I think that's what's so exciting about it - we've never had tools or technologies with this property. It's very different.”

“So you aren't going to this stuff to say ‘you are Shakespeare.’ If you want that you go and read the source.”

“You're going to it to say ‘oh I have a question which might touch, it might be useful to hear from someone who's read a lot of Shakespeare and has some kind of intuition about it.’"

"That's why this whole thing is so exciting and also why people have some appropriate fear about it.”

“Probably the more accurate word is guesstimation. It's guessing but its guesses look a lot like intuitive understanding.”

4️⃣ Combining all of the above, that always available colleague with the “alien intelligence” and unheard of short-term memory can make predictions with previously unimaginable accuracy…

My mind goes to execution velocity, exploring ideas that were previously impossible to ever consider, dramatic reductions in wasted effort etc.

Rick: “Why is predicting the next thing in a sequence so powerful?”

Jack: “Think of it as if you've just moved into this place and there is a leak somewhere and you hear it in the form of a drip in the pipes.”

“Well you'll do all kinds of things to figure out where that leak is but a lot of what you'll be doing is you'll be walking around this building until you have a really good understanding of where all of the pipes are and other things and then you'll sort of reason your way to figuring out where it must be coming from.”

“To do that you've got to be holding a picture in your head of the whole house and everything you understand about it just to figure out this one particular drop and where it's coming from.”

“I think prediction tasks all look like this.”

“To actually make the right prediction in the limit you need to have built an incredibly complex simulation in your head of everything that could be leading to that, every step leading to the key step."

"You need to be able to in your head - it's not just simple steps, it's usually a whole simulation that you can manipulate to be like "well maybe it could be this or maybe it could be this or maybe it could be that."

“It's amazingly powerful.”

“We all underestimate just how much you can get out of this.”

“The types of predictions that this stuff can do are - prediction is the wrong word.”

“Some of it is creative in the same way that some predictions you make come from a creative instinct to allow you to make that prediction.”

“It's not like you're just doing what's probable, you're actually saying 'oh it could also be this as well.' That leads to you making a different prediction.”

Rick: "So could Claude have a gut feeling?”

Jack: “Yes, in the same way that a human can know they want to recommend a certain thing but not be sure exactly why."

"I think this will be true also of AI systems though AI systems may be able to introspect better than people to work out where their gut comes from.”

Rick: “Before the Wright brothers, if AI was trained on human understanding, AI would know that man can't fly and we would never fly.”

Jack: “Exactly."

"We need them to be able to come up with heterodox ideas which are off consensus, against consensus. Allowing systems to be a little out there - that's where creativity comes from.”

5️⃣ If an LLM is a “brain in a jar,” what happens when we give it a body, arms, legs and agency?

More to come on this but the proliferation of interest and announcements around AI agents has been of the “everywhere you look” variety but it’s critical to be very thoughtful about your architecture in this area. Flashy demos, exciting pilots / POCs but scaling to production? Devil is in the details…

Rick: “What do you think is the next step for AI going forward?”

Jack: “Everyone is interested in this idea because it feels very natural and I think of it as we've got the brain in a jar and now we're going to give the brain in a jar arms and legs and say ‘go do stuff.’

“But it also feels like another change that is on the horizon that people inside the AI industry can sense and which will seem really strange and powerful when it happens to people outside.”

“Instead we've trained a general synthetic language agent and now we're trying to give it a life form in the form of an agent. So the ordering is completely the opposite of what I or everyone expected.”

“What we're doing now is we've created a brain and the next step is to create the agent that can live with it.”

Part 2: Now, let’s get to Andrej Karpathy’s lights out presentation to YC.

The conversation between Rick Rubin and Jack Clark does an excellent job providing philosophical and strategic insights.

It's a perfect backdrop and super complimentary for Andrej Karpathy’s technical and practical perspective.

Enterprises that harness this “alien intelligence” by building an ontology aware intelligent system aka operating system will enable always-available, AI subject matter experts to augment human talent and workflows.

Fundamentally changing how knowledge work gets done and establishing the foundation for competitive moat building.

1️⃣ Computers learned our language so we don’t (necessarily) have to learn theirs (i.e. Vibe Coding)

The opportunity to transition from an Application-centric to Data-centric and eventually, Knowledge-centric architecture is in part enabled by the unlocking of human language as the new, most popular coding language.

“We're now programming computers in English. This blew my mind years ago when I tweeted about it - it's currently my pinned tweet."

"At Tesla, we saw functionality that was originally written in Software 1.0 migrated to 2.0. Now we're seeing the same thing again where Software 3.0 is eating through the stack.”

2️⃣ LLMs as Operating Systems

Critical to keep in mind both the "Superpowers" and "Cognitive Deficits" and to design your AI systems accordingly.

“LLMs have very strong analogies to operating systems. They're not just simple commodities like electricity - they're increasingly complex software ecosystems.”

“The LLM is like a new kind of computer - it's the CPU equivalent, context windows are like memory, and the LLM orchestrates memory and compute for problem solving."

"Just like you can download VS Code and run it on Windows, Linux, or Mac, you can take an LLM app like Cursor and run it on GPT, Claude, or Gemini.”

“LLMs flip the direction of technology diffusion."

"Usually, governments and corporations are first users of new expensive technology, then it diffuses to consumers. But with LLMs, it's flipped - consumers got access first."

"My LLM usage is often about 'how do I boil an egg' rather than military ballistics. This informs where the first apps will be.”

“Superpowers”

• “Encyclopedic knowledge and memory”

• “Can remember far more than any individual human”

• “Like Dustin Hoffman's character in Rain Man - perfect memory for vast amounts of information”

“Cognitive Deficits”

• “They hallucinate and make up stuff”

• “Don't have good internal models of self-knowledge”

• “Display 'jagged intelligence' - superhuman in some domains, make mistakes no human would make”

• “Will insist 9.11 is greater than 9.9 or that there are two Rs in ‘strawberry’”

• “Suffer from anterograde amnesia - they don't learn and consolidate knowledge over time like humans do”

3️⃣ Humans + Machines not Humans vs. Machines

I can’t think of anything to add to the next two sentences other than +100000.

“We're cooperating with AIs - they do generation, we do verification. It's in our interest to make this loop as fast as possible.”

“Two Ways to Speed Up the Loop”

1. “Speed up verification - GUIs are extremely important because they utilize your computer vision. Reading text is effortful, but looking at visual representations is a highway to your brain.”

2. “Keep AI on the leash - It's not useful to get a diff of 10,000 lines of code. Even though it comes out instantly, I'm still the bottleneck having to verify it's not introducing bugs or security issues.”

4️⃣ The “Iron Man Suit” analogy might be the next “Vibe Coding” (Andrej invented that term)

Same as #3 so I will just do it again…+100000.

“I love the Iron Man suit analogy. It's both an augmentation (Tony Stark can drive it) and an agent (can fly around autonomously).”

“We want both, but at this stage with fallible LLMs, it's less Iron Man robots and more Iron Man suits.”

“Less flashy demos of autonomous agents, more partial autonomy products with custom GUIs and UI/UX so the generation-verification loop is very fast.”

“We shouldn't lose sight that it's possible to automate this work. There should be an autonomy slider in your product, and you should think about how to slide that slider over time.”

“Going back to the Iron Man suit analogy, over the next decade we're going to take the autonomy slider from left to right. It's going to be very interesting to see what that looks like, and I can't wait to build it with all of you.”

The question isn’t whether AI will transform your software stack, it’s how quickly you can sort out your Buy vs Build vs Partner and how you can start building these Humans + Machines collaboration loops in your organization.

You ready? You've got this.

Get after it.