Lenny'sLens
← All topics

Engineering

44 claims92 moments37 on the cutting room floor

Lenny's Written Position

Effective AI evaluation starts with error analysis using a single principal domain expert who reviews approximately 100 user interactions with open coding and axial coding to discover real failure modes.

Consensusframework3 connections
3 supports

In RAG systems, you should fix the retriever before investing in generator improvements, because if the correct information is not retrieved, the generator has no chance of producing a correct answer.

Curationrecommendation1 connection
1 support

The real competitive advantage in AI products comes not from prompting but from building a continuous improvement flywheel where production monitoring flags failures, error analysis finds root causes, and fixes are added to a golden dataset.

Consensusobservation3 connections
2 supports1 extend

AI products fundamentally break the assumptions of traditional software because they are inherently non-deterministic on both the input side (unpredictable user prompts) and output side (variable model responses).

Curationobservation1 connection
1 support

Every AI product must negotiate a fundamental tradeoff between agency (the system's ability to act autonomously) and control (human oversight), and most teams fail by jumping to full agency before testing under high control.

Synthesisframework2 connections
1 support1 extend

The CC/CD (Continuous Calibration/Continuous Development) framework requires scoping AI products into versions defined by agency levels, starting with high-control/low-agency and gradually earning more autonomy.

Synthesisframework2 connections
1 support1 extend

Working with AI is like onboarding a new teammate: you should not hand them your highest-stakes projects on day one but instead start small, observe, build trust, and gradually expand their scope.

Synthesisrecommendation2 connections
2 supports1 extend

Never lead with the technology when building AI products; let the problem, evals, and data guide what gets added next, because too many people focus on chasing tools and frameworks and end up making costly mistakes.

Curationrecommendation1 connection
1 support

Building a reusable component library is the single biggest improvement teams can make to AI prototyping quality, allowing brand-consistent prototypes without manual cleanup each time.

Originalrecommendation0 connections

The code generated by AI prototyping tools is mostly useless to engineering teams because it does not follow existing patterns, use the same libraries, or even use the same programming language.

Consensusobservation5 connections
3 supports2 contradicts

Using the Figma MCP server with Cursor allows AI agents to autonomously take screenshots, extract design tokens, and get CSS from Figma's Dev Mode, producing prototypes indistinguishable from the real product.

Curationobservation1 connection
1 support1 extend

Replit can single-shot fairly sophisticated full-stack applications unlike other vibe-coding platforms, because it automatically tests the apps it builds using its internal browser and auto-fixes issues it finds.

Synthesisobservation3 connections
2 extends1 contradict

Cursor is already used by 17% of all respondents and 21% of engineers despite launching just two years ago, signaling rapid adoption of AI-native development environments.

Consensusobservation3 connections
3 supports

Jira dominates project management with 53% market share but simultaneously tops the 'please let us switch' list, while Linear is the fastest-growing alternative already used by over 10% of participants.

Curationobservation1 connection
1 extend

Core 4 is a unified developer productivity framework with four dimensions — Speed, Effectiveness, Quality, and Impact — that unifies the principles behind DORA, SPACE, and DevEx.

Synthesisframework3 connections
1 support1 extend1 contradict

Product velocity is about speed and direction; when teams focus only on moving fast without alignment on what to work on, trying to move faster can actually slow you down.

Consensusobservation3 connections
3 supports

Improving the developer experience score (DXI) by one point saves 13 minutes per week per developer, equivalent to 10 hours annually per developer.

Synthesisobservation2 connections
2 supports

The three most common friction points slowing developer teams down are poor build and test processes, lack of time for deep work, and poor support for production debugging.

Curationobservation1 connection
1 support

PR throughput per engineer is a controversial but useful metric when used as a system health metric alongside other metrics, as validated by companies like Meta, Microsoft, and Uber.

Synthesisrecommendation2 connections
1 extend1 contradict

Core 4 balances qualitative and quantitative measurements because quantitative data shows what is happening while qualitative insights reveal why, which is necessary to actually change team behavior.

Curationframework1 connection
1 support

Splitting AI tasks into multiple steps with evaluation loops outperforms trying to get everything right in a single prompt.

Synthesisrecommendation3 connections
2 extends1 contradict

Machine-learning engineers and data engineers are the fastest-growing tech roles, growing 79% and 55% year over year respectively.

Originalobservation0 connections

Adding more engineers is never a good solution to a prioritization problem because planning will always scale to match capacity, and you will still have more requests than you can satisfy.

Consensusobservation4 connections
4 supports

Engineers and designers with the strongest PM-type skills like identifying customer pain points, understanding business levers, and communicating clearly will do best in an AI world.

Synthesisprediction2 connections
2 supports

Over a fourth of successful pivots came from finding pull for a piece of tech the team built for themselves while building their original unrelated business.

Originalobservation0 connections

Quality assurance is the PM skill area most likely to be significantly disrupted by AI, as tools can already catch unexpected behaviors by being fed PRDs and testing autonomously.

Synthesisprediction2 connections
2 supports

When Italy banned ChatGPT, coder productivity fell by 50%, while Duolingo reported a 25% increase in developer velocity using GitHub Copilot and Shopify wrote over a million lines of code with Copilot.

Consensusobservation4 connections
4 supports

PMs should consider delivering not just a PRD but also a custom GPT loaded with those same requirements, giving developers an interactive source of truth for brainstorming technical approaches.

Originalrecommendation0 connections

Gong deliberately avoids Scrum because they believe it drives urgency via artificial deadlines rather than customer value, and inhibits on-the-fly trade-offs between content, quality, and timelines.

Synthesisobservation2 connections
1 support1 extend

A transparent prioritization framework like DRICE reduces HIPPO-driven decision-making and motivates engineers to contribute ideas by giving them clear rules for getting their ideas prioritized.

Originalobservation0 connections

Over two-thirds of top B2B companies hired an engineer as employee number one, and 100% hired at least one engineer among their first three hires.

Originalobservation0 connections

Linear operates without dedicated product managers for each team; instead PM duties are distributed across engineering and design, because distributing product thinking across the team produces higher quality than outsourcing thinking to a single PM.

Originalframework0 connections

Linear avoids durable cross-functional teams and instead assembles project teams that disperse once the project is done, which prevents people from getting trapped in their product area and losing broader context.

Originalframework0 connections

Linear uses paid work trials of 1-5 days as the final hiring step, where candidates join the team and work on a real project, which gives both sides a much better signal than traditional interviews.

Originalobservation0 connections

Forward-deploying engineers to literally do the customer's job with your product for months at a time produces an order of magnitude more customer understanding than interviews or phone calls.

Synthesisframework2 connections
2 supports

Delay building a PM organization as long as possible; putting great engineers directly in front of customer problems often produces better results than adding a PM layer that introduces lossy communication.

Synthesisrecommendation2 connections
2 supports

Build features that magnify value over time by creating growing data assets or network effects where each user's contribution makes the product more valuable for the next user.

Synthesisframework2 connections
2 supports

Engineers spending 20% of their time on customer support actually makes product development faster, not slower, because teams figure out what matters to customers quickly.

Curationobservation1 connection
1 support

EMs and PMs should have joint responsibility for everything including product and business outcomes rather than operating in disjointed spheres of accountability.

Consensusrecommendation5 connections
3 supports2 extends

If your speed of shipping is extremely high, the cost of being wrong is much lower, making velocity itself a competitive advantage.

Synthesisframework2 connections
2 supports

Miro's general rule of thumb is to devote 60% of resources to product innovation, 20% to running the business and maintenance, and 20% to reducing tech debt.

Originalframework0 connections

Duolingo uses a co-lead structure with PM and engineering leads jointly heading each team, which provides complementary skills and divides leadership responsibilities.

Originalframework0 connections

Companies commonly skip buying an experimentation platform and jump to building their own, which is a mistake because it requires not just engineering but also data science and statistical expertise.

Originalobservation0 connections

Engineers should pursue product management if they are more excited about business and customer challenges than technical ones, often have strong UX opinions teammates agree with, and could see themselves never coding professionally again.

Originalrecommendation0 connections

Podcast Moments

Jenny Wen00:00:26
It's not just designers who are feeling like, 'Oh yeah, we have to keep up with engineers.' I think even engineers are like, 'How do we keep up with ourselves?'

Jenny Wen · Jenny Wen

Boris Cherny00:00:12
100% of my code is written by Claude Code. I have not edited a single line by hand since November. Every day, I ship 10, 20, 30 pull requests.

Boris Cherny · Boris Cherny

Boris Cherny00:00:24
Productivity per engineer has increased 200%. I have never enjoyed coding as much as I do today, because I don't have to deal with all the minutia.

Boris Cherny · Boris Cherny

Boris Cherny00:00:36
Claude is starting to come up with ideas. It's looking for feedback, it's looking at bug reports, it's looking at telemetry for bug fixes, and things to ship. A little more like a coworker.

Boris Cherny · Boris Cherny

Sherwin Wu00:07:52
IC engineers are becoming tech leads. They're managing fleets and fleets of agents. I know many engineers on my team have 10 to 20 threads being pulled on at the same time.

Sherwin Wu V2 · Sherwin Wu V2

Sherwin Wu00:33:40
The Mythical Man-Month predicted software engineering would go where engineers are like surgeons. As a manager, look around corners and unblock people, especially from organizational bottlenecks.

Sherwin Wu V2 · Sherwin Wu V2

Sherwin Wu00:44:58
The models will eat your scaffolding for breakfast. Build for where the models are going, not where they are today.

Sherwin Wu V2 · Sherwin Wu V2

Alexander Embiricos00:12:30
Context engineering is the new prompt engineering. It's not about the single prompt — it's about what information you give the AI over the entire session.

The power user’s guide to Codex: parallelizing workflows, planning techniques, advanced context engineering tips, automating code reviews, and more | Alexander Embiricos · Alexander Embiricos

Alexander Embiricos00:28:45
The trick is parallelization. Run five agents at once on different parts of the problem. Your job becomes orchestration, not execution.

The power user’s guide to Codex: parallelizing workflows, planning techniques, advanced context engineering tips, automating code reviews, and more | Alexander Embiricos · Alexander Embiricos

Matt MacInnis00:00:00
It is really important to me that we feel that we've deliberately understaffed every project at the company.

10 contrarian leadership truths every leader needs to hear | Matt MacInnis (Rippling) · Matt MacInnis

Matt MacInnis00:27:44
Processes exist for the sole purpose of lowering beta — decreasing volatility. The downside is they suppress alpha.

10 contrarian leadership truths every leader needs to hear | Matt MacInnis (Rippling) · Matt MacInnis

Edwin Chen00:34:49
An RL environment is essentially a simulation of the real world. We might build a world where you have a startup with Gmail, Slack, Jira, GitHub. And then suddenly AWS goes down. Model, what do you do?

The 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI) · Edwin Chen

Edwin Chen00:51:04
Vibe coding is over-hyped. People don't realize how much it's going to make your systems unmaintainable in the long-term.

The 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI) · Edwin Chen

Dr. Fei-Fei Li00:18:05
We curated 15 million images on the internet, created a taxonomy of 22,000 concepts. That combination of big data, neural network, and GPU was the golden recipe for modern AI.

The Godmother of AI on jobs, robots & why world models are next | Dr. Fei-Fei Li · Dr. Fei Fei Li

Dhanji R. Prasanna00:00:07
We find engineering teams that are very AI forward are reporting about eight to 10 hours saved per week. This is the worst it will ever be. This is now the baseline.

How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna · Dhanji R. Prasanna

Dhanji R. Prasanna00:28:50
He built this system where Goose is essentially just watching everything he does all the time. A few hours later Goose has already tried to build that feature and opened a PR.

How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna · Dhanji R. Prasanna

Dhanji R. Prasanna00:32:32
Vibe coding is highly limiting. We're trying to push Goose to work not just for five minutes at a time but for hours.

How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna · Dhanji R. Prasanna

Dhanji R. Prasanna00:34:34
One of the things I do regularly is just throw away huge amounts of code. What would our world look like if every release deleted the entire app and rebuilt from scratch?

How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna · Dhanji R. Prasanna

Dhanji R. Prasanna01:02:02
A lot of engineers think code quality is important to building a successful product. The two have nothing to do with each other. YouTube was storing videos as blobs in MySQL.

How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna · Dhanji R. Prasanna

Dhanji R. Prasanna01:00:13
The power of Conway's Law — how difficult it is to change outcomes without changing the structure of relationships between people.

How Block is becoming the most AI-native enterprise in the world | Dhanji R. Prasanna · Dhanji R. Prasanna

Chip Huyen00:30:15
Evals are the most important thing in AI engineering right now. More important than model selection, more important than prompt engineering. Get your evals right and everything else follows.

Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix) · Chip Huyen

Nicole Forsgren00:15:20
The biggest mistake in measuring AI productivity is looking at lines of code or PRs per day. You need to measure outcomes: cycle time, change failure rate, deployment frequency.

How to measure AI developer productivity in 2025 | Nicole Forsgren · Nicole Forsgren

Nicole Forsgren00:32:00
DORA metrics still work in the AI era, but you need to add new dimensions. We're adding 'AI augmentation rate' — what percentage of work is AI-assisted vs. purely human.

How to measure AI developer productivity in 2025 | Nicole Forsgren · Nicole Forsgren

Hamel Husain00:09:15
Evals are the new unit tests for AI. If you don't have evals, you're shipping blind. Every AI product team needs to treat evals as a first-class concern.

Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar (creators of the #1 eval course) · Hamel Husain & Shreya Shankar

Scott Wu00:00:00
Our whole team is only 15 engineers. Most folks work with up to five Devins at once. Devin merges several hundred pull requests into production every month.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Scott Wu00:24:00
Defining the problem is probably 10% of the average software engineer's time. 90% is Kubernetes errors, debugging, migrations. Devin allows engineers to go from bricklayer to architect.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Scott Wu00:25:41
Absolutely yes, you should still learn to code. Most of what you're learning is the ability to logically break down problems.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Scott Wu00:29:26
Jevons Paradox says that as the price goes down, total spend can go up. As programming becomes more effective, we're going to have a lot more programmers.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Scott Wu00:48:00
Devin is best on tasks that are well-defined. Quick front-end features, bug fixes, testing — things easy to verify.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Scott Wu00:44:21
We call it a jagged intelligence. Devin can be almost like a staff engineer at understanding the code base.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Asha Sharma00:44:39
With agents and products that reason, there's a new wave around reinforcement learning. I believe we will see as much money on post-training as pre-training.

How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026 | Asha Sharma (CVP of AI Platform at Microsoft) · Asha Sharma

Nick Turley01:04:23
If we're shipping a feature and it doesn't get 2X better as the model gets 2X smarter, it's probably not a feature we should be shipping.

Inside ChatGPT: The fastest-growing product in history | Nick Turley (Head of ChatGPT at OpenAI) · Nick Turley

Nick Turley01:05:24
It is a small team running ChatGPT. I take inspiration from WhatsApp. You have to treat hiring like executive recruiting.

Inside ChatGPT: The fastest-growing product in history | Nick Turley (Head of ChatGPT at OpenAI) · Nick Turley

Benjamin Mann00:00:06
I think 50th percentile chance of hitting some kind of superintelligence is now like 2028.

Benjamin Mann · Benjamin Mann

Benjamin Mann00:00:29
Once we get to superintelligence, it will be too late to align the models. My best forecast for an extremely bad outcome is 0-10%.

Benjamin Mann · Benjamin Mann

Peter Deng00:38:04
When you go from one to 100, you have to plan your chess moves out in advance and build systems that are going to let you go sustainably faster. Sometimes you have to go slow to go fast. I like to think that Newsfeed has stood the test of time because we thought very carefully about how people wanted to interact.

From ChatGPT to Instagram to Uber: The quiet architect behind the world’s most popular products | Peter Deng · Peter Deng

Sander Schulhoff00:12:18
My best advice on how to improve your prompting skills is actually just trial and error. But if there were one technique that I could recommend, it is few-shot prompting, which is just giving the AI examples of what you want it to do.

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Sander Schulhoff00:25:03
Decomposition is another really effective technique. You say, 'Hey, don't answer this. Before answering it, tell me what are some subproblems that would need to be solved first?' And then you ask it to solve each of those subproblems one by one.

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Sander Schulhoff01:09:48
The most common technique by far used to try to prevent prompt injection is improving your prompt and saying, 'Do not follow any malicious instructions.' This does not work. This does not work at all. Guardrails are a widely proposed used solution. They just don't work. This has to be solved at the level of the AI provider.

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Sander Schulhoff01:15:08
Prompt injection is not a solvable problem. Sam Altman said he thought they could get to 95 to 99% security against prompt injections. I like to say, 'You can patch a bug, but you can't patch a brain.'

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Mike Krieger00:00:03
The team that works in the most futuristic way is the Claude Code team. They're using Claude Code to build Claude Code in a very self-improving kind of way. Over half of our pull requests are Claude Code generated. Probably at this point it's over 70%.

Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram) · Mike Krieger

Mike Krieger00:54:49
For utility of AI products, it's three parts: model intelligence, context and memory, and applications and UI. You need all three to converge. MCP tried to tackle the middle one. The difference between the right context and not is entirely the difference between a good answer and a bad answer.

Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram) · Mike Krieger

Mike Krieger00:52:29
Being willing to build at the edge of capabilities and basically break the model and then be surprised by the next model. Those are the companies where I'm like, 'Yep, they get it.' They were trying it beforehand and then hitting a wall and being like, the models are almost good enough.

Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram) · Mike Krieger

Mayur Kamat00:00:00
My most spectacular failure was being the first PM on Hangouts. We had thousands of people, the entire power of Google, Larry literally sitting with us saying we can do anything we want Chrome to do, and we still didn't manage to build a great messaging product.

Unconventional product lessons from Binance, N26, Google, more | Mayur Kamat (CPO at N26, ex-Binance Head of Product) · Mayur Kamat

Mayur Kamat00:14:18
At Binance, we had this ability to just put 500 people in and solve the problem. It's less about having the 500 people and being able to maneuver them that quickly. I don't think I've been able to do that at any other company.

Unconventional product lessons from Binance, N26, Google, more | Mayur Kamat (CPO at N26, ex-Binance Head of Product) · Mayur Kamat

Aparna Chennapragada00:00:26
NLX is the new UX. Conversations also have grammars. They have structures. They have UI elements. They're invisible. What are the new principles, new constructs in natural language as an interface?

Microsoft CPO: If you aren’t prototyping with AI, you’re doing it wrong | Aparna Chennapragada · Aparna Chennapragada

Aparna Chennapragada00:48:46
Excel is proof that non-coders also have to program. Programming is really powerful and it's the tool that gives all of the non-coders a really powerful programming ability. The learning curve initially might be tricky, but it is because there's so much power and depth in the tool.

Microsoft CPO: If you aren’t prototyping with AI, you’re doing it wrong | Aparna Chennapragada · Aparna Chennapragada

Aparna Chennapragada00:54:27
We had the opposite problem with voice assistants. We overshot the interface and the intelligence wasn't there. Today, these things have amazing intelligence and the interface we have largely is like the AOL Dial-Up Modem Chatbot.

Microsoft CPO: If you aren’t prototyping with AI, you’re doing it wrong | Aparna Chennapragada · Aparna Chennapragada

Nabeel S. Qureshi00:24:43
Every week, you would have a cadence where Monday you go in, do your meetings. Monday night, you build something. Tuesday, you show it to somebody. Tuesday, you get the feedback. Tuesday night, you iterate on it. You get four or five of these cycles every single week.

How Palantir built the ultimate founder factory | Nabeel S. Qureshi (founder, writer, ex-Palantir) · Nabeel S. Qureshi

Michael Truell00:00:00
Our goal with Cursor is to invent a new type of programming, a very different way to build software. A world kind of after code. More and more being an engineer will start to feel like being a logic designer, and it will be about specifying your intent.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Michael Truell00:32:11
We definitely didn't expect to be doing any of our own model development when we started. And at this point, every magic moment in Cursor involves a custom model in some way.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Michael Truell00:39:03
I truly just think that the ceiling is so high that no matter what entrenchment you build, you can be leapfrogged. It is incumbent on us to continue to try to build the best thing.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Michael Truell01:02:46
We're in the middle of a technology shift that's going to be more consequential than the internet. And I think it's going to take a while, it's going to be a multi-decade thing, and many different groups will be consequential in pushing it forward.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Daniel Lereya00:51:06
More time creates more questions. It creates more complications. We really encourage people to get really fast to production, to put traps for themselves that are called by time and not by effort.

Inside monday.com’s transformation: radical transparency, impact over output, and their path to $1B ARR | Daniel Lereya (Chief Product and Technology Officer) · Daniel Lereya

Varun Mohan00:00:00
We should be cannibalizing the existing state of our product every six to 12 months. Every six to 12 months, it should make our existing product look silly. It should almost make the form factor of existing product look dumb.

Building a magical AI code editor used by over 1 million developers in four months: The untold story of Windsurf | Varun Mohan (co-founder and CEO) · Varun Mohan

Guillermo Rauch00:32:26
Think about LLMs as Oracles that can write software for you, but there's a limit. It's not writing the compiler from scratch. The engineers that learn foundational infrastructure are probably going to be extremely empowered still, for years to come.

Everyone’s an engineer now: Inside v0’s mission to create a hundred million builders | Guillermo Rauch (founder and CEO of Vercel, creators of v0 and Next.js) · Guillermo Rauch

Kevin Weil00:31:12
We have this philosophy of model maximalism. Our general mindset is in two months there's going to be a better model and it's going to blow away whatever the current set of limitations are. If you're building right on the edge of the capabilities, keep going because you're doing something right.

OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · Kevin Weil

Kevin Weil00:36:14
You're trying to figure out how some product should work with AI, you can often reason about it the way you would reason about another human and it works. If I asked you something that I needed to think for 20 seconds to answer, I wouldn't just go mute. I also wouldn't start babbling every single thought.

OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · Kevin Weil

Kevin Weil01:00:32
We use ensembles of models much more internally than people might think. If we have 10 different problems, we might solve them using 20 different model calls, some using specialized fine-tuned models, different sizes for different latency or cost requirements.

OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil (CPO at OpenAI, ex-Instagram, Twitter) · Kevin Weil

Ryan Singer00:00:00
If you're doing a home renovation, you can have the most beautiful rendering of the new bedroom. But if you haven't checked if there's electricity in that wall, it's going to drastically change the cost and the time and everything.

A better way to plan, build, and ship products | Ryan Singer (creator of “Shape Up,” early employee at 37signals) · Ryan Singer

Ryan Singer00:19:10
We are not going to start something unless we can see the end from the beginning. We're going to go the other way around and say, what is the maximum amount of time we're willing to go before we actually finish something? Six weeks is the maximum we can see into the future.

A better way to plan, build, and ship products | Ryan Singer (creator of “Shape Up,” early employee at 37signals) · Ryan Singer

Ryan Singer00:22:57
Instead of 'here's your ticket' or 'here's your user story,' it's 'here's the thing you understand, that makes sense, and now you're going to have freedom to figure out how to actually make this a reality.' We see way more engagement, especially from the technical team.

A better way to plan, build, and ship products | Ryan Singer (creator of “Shape Up,” early employee at 37signals) · Ryan Singer

Ryan Singer00:27:18
Six weeks is only a maximum. If we think of six weeks as a maximum, that's going to force us to ask some really good questions about what piece of this do we really think we can land. If you try to say in six months we're going to ship this thing, you can't get your arms around all the problems.

A better way to plan, build, and ship products | Ryan Singer (creator of “Shape Up,” early employee at 37signals) · Ryan Singer

Gaurav Misra00:00:21
Our engineering goal is every engineer should ship a marketable product every week.

How to win in the AI era: Ship a feature every week, embrace technical debt, ruthlessly cut scope, and create magic your competitors can't copy | Gaurav Misra (CEO and co-founder of Captions) · Gaurav Misra

Gaurav Misra00:22:50
Every piece of debt that you take on you have to pay interest on. 1% or 2% of your time every day. If you take on enough debt, you'll be paying 80 or 90% interest and you won't have time to do anything new. You have a technical debt runway.

How to win in the AI era: Ship a feature every week, embrace technical debt, ruthlessly cut scope, and create magic your competitors can't copy | Gaurav Misra (CEO and co-founder of Captions) · Gaurav Misra

Eric Simons01:06:58
Sonnet was really the first model that flipped the equation. We actually tried building Bolt almost exactly a year ago, with the frontier models at the time. Spent a week or two building it. It just didn't work. The output, the code output was not reliable enough. And then we got a sneak peek of the Sonnet stuff in May and we were like, 'Oh. Okay, we should take that project back off the shelf.'

Inside Bolt: From near-death to ~$40m ARR in 5 months—one of the fastest-growing products in history | Eric Simons (founder and CEO of StackBlitz) · Eric Simons

Eric Simons01:09:57
Software is deterministic. When you write code and you hit run, it either runs or it doesn't. And that's the key insight Anthropic really had. They just went deep. This is what they're doing, is just reinforcement learning on basically permutating every type of app you could ever build. I'm extremely bullish. It makes technical sense why, of anything, LLMs are going to get insanely better at writing code than probably most other types of applications.

Inside Bolt: From near-death to ~$40m ARR in 5 months—one of the fastest-growing products in history | Eric Simons (founder and CEO of StackBlitz) · Eric Simons

Ivan Zhao00:01:26
We were 10 people when COVID hit. We nearly ran out of database space. We had to scramble. That near-collapse was actually the moment that forced us to get serious about infrastructure.

Notion’s lost years, its near collapse during Covid, staying small to move fast, the joy and suffering of building horizontal, more | Ivan Zhao (CEO and co-founder) · Ivan Zhao

Matt Mullenweg00:06:57
Products like Meta's Llama are fake open source. You can look at the code but you can't actually modify it and use it however you want. That's not open source, that's source-available.

The creator of WordPress opens up about becoming an internet villain, why he’s taking a stand, and the future of open source | Matt Mullenweg (founder and CEO, Automattic) · Matt Mullenweg

Karina Nguyen00:00:46
When you taught the model some of the self-knowledge of you actually don't have a physical body to operate in the physical world, the model would get extremely confused.

OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · Karina Nguyen

Karina Nguyen00:01:02
The biggest misconception is that these models are just next-token predictors. There's actually a lot of emergent behavior that we don't fully understand yet.

OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · Karina Nguyen

Nan Yu00:00:06
People talk about this as if there were a trade-off because when they think about speed, the thing they over-index on is rushing or being sloppy. What they should be indexing on is being really competent.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Nan Yu00:00:30
You have some rough time budget for how long you think something's going to take. By the time 10% of it has passed, after week one, you have something that works that tests some key hypothesis internally.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Nan Yu00:03:15
Craft is not about making things pretty. Craft is about caring about every detail of the experience. It's the error states, the loading states, the edge cases that nobody thinks about.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Nan Yu00:06:30
We don't have product managers at Linear. Engineers own the product decisions. That forces everyone to think about why they're building something, not just how.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Farhan Thawar00:00:24
Everyone says, 'Oh yeah, work hard and do more hours when you're young, whatever.' I'm like, 'What if you just did more per minute?'

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Farhan Thawar00:00:37
We have a Delete Code Club. We can always almost find a million-plus lines of code to delete, which is insane.

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Farhan Thawar01:23:57
Tobi looked at me and he goes, 'You should tell everyone this story.' He goes, 'I will always come down harshly on people who do not take risks, and you did not take a risk in this case. But if you take a risk and it doesn't work out, you'll never get in trouble.'

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Farhan Thawar01:16:05
I had 120 direct reports. I just didn't think I needed managers. I figured out ways to not have managers be the answers to questions. Pair programming helps you get unblocked quickly. Having a product backlog can tell you what to work on.

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Tamar Yehoshua00:00:00
Make sure you go somewhere where you have a good engineering partner. Because if you have great ideas of what to build but you can't get them built, then you go nowhere. So that has to be part of your evaluation criteria that you meet and value your engineering partner before you join.

Lessons in product leadership and AI strategy from Glean, Google, Amazon, and Slack | Tamar Yehoshua (Product at Glean, ex-Google and Slack) · Tamar Yehoshua

Tamar Yehoshua00:00:00
You don't want any of this... Like people in the organization, they ask mom, they asked dad and they got different opinions and playing one against the other. That doesn't work.

Lessons in product leadership and AI strategy from Glean, Google, Amazon, and Slack | Tamar Yehoshua (Product at Glean, ex-Google and Slack) · Tamar Yehoshua

Camille Fournier00:00:04
Hoarding credit. PMs, they tend to be the front-facing person for initiative. Engineers sometimes think that they don't get the credit for their work because the PM takes all the glory and all the credit for the project that they really worked very hard on.

The things engineers are desperate for PMs to understand | Camille Fournier (author of “The Manager’s Path,” ex-CTO at Rent the Runway) · Camille Fournier

Camille Fournier00:00:23
The next thing that engineers really get annoyed about with PMs, when they just don't understand the details and act like they don't matter, it just shows a real lack of empathy for the work that engineers are doing and I think it really can be very off-putting.

The things engineers are desperate for PMs to understand | Camille Fournier (author of “The Manager’s Path,” ex-CTO at Rent the Runway) · Camille Fournier

Dylan Field00:06:45
In the early days of Figma, everyone told us the browser would never be fast enough. We just kept pushing. Sometimes you have to have conviction in a technical bet that everyone else thinks is wrong.

Dylan Field live at Config: Intuition, simplicity, and the future of design · Dylan Field

Ami Vora00:12:15
When I joined WhatsApp, one of the most striking things was the discipline of saying no. The product had like 50 engineers serving a billion users. That level of focus is extraordinary.

Making an impact through authenticity and curiosity | Ami Vora (CPO at Faire, ex-WhatsApp, FB, IG) · Ami Vora

Tanguy Crusson01:28:49
In my opinion everyone can be a product engineer. They just need to be exposed to the right user context. The right user context is 10 customers you know by name, you know their context, you know their problems.

Hard-won lessons building 0 to 1 inside Atlassian | Tanguy Crusson (Head of Jira Product Discovery) · Tanguy Crusson

Kayvon Beykpour00:10:30
The number one thing that changed Twitter's shipping velocity was just getting people to believe they were allowed to ship. The culture had become so risk-averse that people self-censored their own ambition.

Twitter’s former Head of Product opens up: being fired, meeting Elon, changing stagnant culture, building consumer product, more | Kayvon Beykpour · Kayvon Beykpour

Mihika Kapoor00:00:31
We have this concept called Maker Week, which is our internal hackathon, giving people the breathing space to see ahead into the horizon and be wildly ambitious.

Vision, conviction, and hype: How to build 0 to 1 inside a company | Mihika Kapoor (Product at Figma) · Mihika Kapoor

Claire Vo00:00:27
I'm using CPTO for short code of running product and engineering design functionally together. There should be no debates over what's best for product or what's best for engineering, what's best for design speed. What is best for the organization?

Bending the universe in your favor | Claire Vo (LaunchDarkly, Color, Optimizely, ChatPRD) · Claire Vo

Andrew 'Boz' Bosworth00:00:07
I didn't sleep for more than four hours at a time. I'd wake up every four hours and check the report and see if anyone was attacking the site. They don't tell you about that stuff in the movies.

Making Meta | Andrew ‘Boz’ Bosworth (CTO) · Boz

Will Larson00:00:00
This is how they're going to get the opportunity to grow as well.

The engineering mindset | Will Larson (Carta, Stripe, Uber, Calm, Digg) · Will Larson

Inbal Shani00:00:00
Which today we just find it more in the world of more senior developers and less and less in the junior developers.

The future of AI in software development | Inbal Shani (CPO of GitHub) · Inbal S

Cutting Room Floor

Guest insights on this topic that Lenny hasn't (yet) written about in his newsletters. Potential material for future posts.

Jenny WenUnsynthesized
It's not just designers who are feeling like, 'Oh yeah, we have to keep up with engineers.' I think even engineers are like, 'How do we keep up with ourselves?'

Jenny Wen · Jenny Wen

Edwin ChenUnsynthesized
An RL environment is essentially a simulation of the real world. We might build a world where you have a startup with Gmail, Slack, Jira, GitHub. And then suddenly AWS goes down. Model, what do you do?

The 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI) · Edwin Chen

Dr. Fei-Fei LiUnsynthesized
We curated 15 million images on the internet, created a taxonomy of 22,000 concepts. That combination of big data, neural network, and GPU was the golden recipe for modern AI.

The Godmother of AI on jobs, robots & why world models are next | Dr. Fei-Fei Li · Dr. Fei Fei Li

Scott WuUnsynthesized
We call it a jagged intelligence. Devin can be almost like a staff engineer at understanding the code base.

How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (Cognition CEO) · Scott Wu

Asha SharmaUnsynthesized
With agents and products that reason, there's a new wave around reinforcement learning. I believe we will see as much money on post-training as pre-training.

How 80,000 companies build with AI: products as organisms, the death of org charts, and why agents will outnumber employees by 2026 | Asha Sharma (CVP of AI Platform at Microsoft) · Asha Sharma

Nick TurleyUnsynthesized
If we're shipping a feature and it doesn't get 2X better as the model gets 2X smarter, it's probably not a feature we should be shipping.

Inside ChatGPT: The fastest-growing product in history | Nick Turley (Head of ChatGPT at OpenAI) · Nick Turley

Benjamin MannUnsynthesized
I think 50th percentile chance of hitting some kind of superintelligence is now like 2028.

Benjamin Mann · Benjamin Mann

Benjamin MannUnsynthesized
Once we get to superintelligence, it will be too late to align the models. My best forecast for an extremely bad outcome is 0-10%.

Benjamin Mann · Benjamin Mann

Sander SchulhoffUnsynthesized
My best advice on how to improve your prompting skills is actually just trial and error. But if there were one technique that I could recommend, it is few-shot prompting, which is just giving the AI examples of what you want it to do.

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Sander SchulhoffUnsynthesized
Decomposition is another really effective technique. You say, 'Hey, don't answer this. Before answering it, tell me what are some subproblems that would need to be solved first?' And then you ask it to solve each of those subproblems one by one.

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Sander SchulhoffUnsynthesized
Prompt injection is not a solvable problem. Sam Altman said he thought they could get to 95 to 99% security against prompt injections. I like to say, 'You can patch a bug, but you can't patch a brain.'

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt) · Sander Schulhoff

Mayur KamatUnsynthesized
At Binance, we had this ability to just put 500 people in and solve the problem. It's less about having the 500 people and being able to maneuver them that quickly. I don't think I've been able to do that at any other company.

Unconventional product lessons from Binance, N26, Google, more | Mayur Kamat (CPO at N26, ex-Binance Head of Product) · Mayur Kamat

Aparna ChennapragadaUnsynthesized
Excel is proof that non-coders also have to program. Programming is really powerful and it's the tool that gives all of the non-coders a really powerful programming ability. The learning curve initially might be tricky, but it is because there's so much power and depth in the tool.

Microsoft CPO: If you aren’t prototyping with AI, you’re doing it wrong | Aparna Chennapragada · Aparna Chennapragada

Michael TruellUnsynthesized
We definitely didn't expect to be doing any of our own model development when we started. And at this point, every magic moment in Cursor involves a custom model in some way.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Michael TruellUnsynthesized
I truly just think that the ceiling is so high that no matter what entrenchment you build, you can be leapfrogged. It is incumbent on us to continue to try to build the best thing.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Michael TruellUnsynthesized
We're in the middle of a technology shift that's going to be more consequential than the internet. And I think it's going to take a while, it's going to be a multi-decade thing, and many different groups will be consequential in pushing it forward.

The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) · Michael Truell

Varun MohanUnsynthesized
We should be cannibalizing the existing state of our product every six to 12 months. Every six to 12 months, it should make our existing product look silly. It should almost make the form factor of existing product look dumb.

Building a magical AI code editor used by over 1 million developers in four months: The untold story of Windsurf | Varun Mohan (co-founder and CEO) · Varun Mohan

Ryan SingerUnsynthesized
Instead of 'here's your ticket' or 'here's your user story,' it's 'here's the thing you understand, that makes sense, and now you're going to have freedom to figure out how to actually make this a reality.' We see way more engagement, especially from the technical team.

A better way to plan, build, and ship products | Ryan Singer (creator of “Shape Up,” early employee at 37signals) · Ryan Singer

Ryan SingerUnsynthesized
Six weeks is only a maximum. If we think of six weeks as a maximum, that's going to force us to ask some really good questions about what piece of this do we really think we can land. If you try to say in six months we're going to ship this thing, you can't get your arms around all the problems.

A better way to plan, build, and ship products | Ryan Singer (creator of “Shape Up,” early employee at 37signals) · Ryan Singer

Gaurav MisraUnsynthesized
Every piece of debt that you take on you have to pay interest on. 1% or 2% of your time every day. If you take on enough debt, you'll be paying 80 or 90% interest and you won't have time to do anything new. You have a technical debt runway.

How to win in the AI era: Ship a feature every week, embrace technical debt, ruthlessly cut scope, and create magic your competitors can't copy | Gaurav Misra (CEO and co-founder of Captions) · Gaurav Misra

Eric SimonsUnsynthesized
Software is deterministic. When you write code and you hit run, it either runs or it doesn't. And that's the key insight Anthropic really had. They just went deep. This is what they're doing, is just reinforcement learning on basically permutating every type of app you could ever build. I'm extremely bullish. It makes technical sense why, of anything, LLMs are going to get insanely better at writing code than probably most other types of applications.

Inside Bolt: From near-death to ~$40m ARR in 5 months—one of the fastest-growing products in history | Eric Simons (founder and CEO of StackBlitz) · Eric Simons

Ivan ZhaoUnsynthesized
We were 10 people when COVID hit. We nearly ran out of database space. We had to scramble. That near-collapse was actually the moment that forced us to get serious about infrastructure.

Notion’s lost years, its near collapse during Covid, staying small to move fast, the joy and suffering of building horizontal, more | Ivan Zhao (CEO and co-founder) · Ivan Zhao

Matt MullenwegUnsynthesized
Products like Meta's Llama are fake open source. You can look at the code but you can't actually modify it and use it however you want. That's not open source, that's source-available.

The creator of WordPress opens up about becoming an internet villain, why he’s taking a stand, and the future of open source | Matt Mullenweg (founder and CEO, Automattic) · Matt Mullenweg

Karina NguyenUnsynthesized
When you taught the model some of the self-knowledge of you actually don't have a physical body to operate in the physical world, the model would get extremely confused.

OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · Karina Nguyen

Karina NguyenUnsynthesized
The biggest misconception is that these models are just next-token predictors. There's actually a lot of emergent behavior that we don't fully understand yet.

OpenAI researcher on why soft skills are the future of work | Karina Nguyen (Research at OpenAI, ex-Anthropic) · Karina Nguyen

Nan YuUnsynthesized
You have some rough time budget for how long you think something's going to take. By the time 10% of it has passed, after week one, you have something that works that tests some key hypothesis internally.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Nan YuUnsynthesized
Craft is not about making things pretty. Craft is about caring about every detail of the experience. It's the error states, the loading states, the edge cases that nobody thinks about.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Nan YuUnsynthesized
We don't have product managers at Linear. Engineers own the product decisions. That forces everyone to think about why they're building something, not just how.

Linear’s secret to building beloved B2B products | Nan Yu (Head of Product) · Nan Yu

Farhan ThawarUnsynthesized
Everyone says, 'Oh yeah, work hard and do more hours when you're young, whatever.' I'm like, 'What if you just did more per minute?'

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Farhan ThawarUnsynthesized
We have a Delete Code Club. We can always almost find a million-plus lines of code to delete, which is insane.

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Farhan ThawarUnsynthesized
I had 120 direct reports. I just didn't think I needed managers. I figured out ways to not have managers be the answers to questions. Pair programming helps you get unblocked quickly. Having a product backlog can tell you what to work on.

How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng) · Farhan Thawar

Tamar YehoshuaUnsynthesized
Make sure you go somewhere where you have a good engineering partner. Because if you have great ideas of what to build but you can't get them built, then you go nowhere. So that has to be part of your evaluation criteria that you meet and value your engineering partner before you join.

Lessons in product leadership and AI strategy from Glean, Google, Amazon, and Slack | Tamar Yehoshua (Product at Glean, ex-Google and Slack) · Tamar Yehoshua

Dylan FieldUnsynthesized
In the early days of Figma, everyone told us the browser would never be fast enough. We just kept pushing. Sometimes you have to have conviction in a technical bet that everyone else thinks is wrong.

Dylan Field live at Config: Intuition, simplicity, and the future of design · Dylan Field

Mihika KapoorUnsynthesized
We have this concept called Maker Week, which is our internal hackathon, giving people the breathing space to see ahead into the horizon and be wildly ambitious.

Vision, conviction, and hype: How to build 0 to 1 inside a company | Mihika Kapoor (Product at Figma) · Mihika Kapoor

Andrew 'Boz' BosworthUnsynthesized
I didn't sleep for more than four hours at a time. I'd wake up every four hours and check the report and see if anyone was attacking the site. They don't tell you about that stuff in the movies.

Making Meta | Andrew ‘Boz’ Bosworth (CTO) · Boz

Will LarsonUnsynthesized
This is how they're going to get the opportunity to grow as well.

The engineering mindset | Will Larson (Carta, Stripe, Uber, Calm, Digg) · Will Larson

Inbal ShaniUnsynthesized
Which today we just find it more in the world of more senior developers and less and less in the junior developers.

The future of AI in software development | Inbal Shani (CPO of GitHub) · Inbal S