No One Knows Anything About AI

I want to present you with two narratives about AI. Both of them are about using this technology to automate computer programming, but they point toward two very different conclusions.

The first narrative notes that Large Language Models (LLMs) are exceptionally well-suited for coding because source code, at its core, is just very well-structured text, which is exactly what these models excel at generating. Because of this tight match between need and capability, the programming industry is serving as an economic sacrificial lamb, the first major sector to suffer a major AI-driven upheaval.

There has been no shortage of evidence to support these claims. Here are some examples, all from the last two months:

• Aravind Srinivas, the CEO of the AI company Perplexity, ​claims​ AI tools like Cursor and GitHub Copilot cut task completion time for his engineers from “three or four days to one hour.” He now mandates every employee in his company to use them: “The speed at which you can fix bugs and ship to production is scary.”
• ​An article in Inc​. confidently declared: “In the world of software engineering, AI has indeed changed everything.”
• Not surprisingly, these immense new capabilities are being blamed for dire disruptions. ​One article​ from an investment site featured an alarming headline: “Tech Sector Sees 64,000 Job Cuts This Year Due to AI Advancement.” No one is safe from such cuts. “Major companies like Microsoft have been at the forefront of these layoffs,” the article explains, “citing AI advancements as a primary factor.”
• My world of academic computer science hasn’t been spared either. A ​splashy Atlantic piece​ opens with a distressing claim: “The Computer Science-Bubble is Bursting,” which it largely blames on AI, a technology it describes as “ideally suited to replace the very type of person who built it.”

Given the confidence of these claims, you’d assume that computer programmers are rapidly going the way of the telegraph operator. But, if you read a different set of articles and quotes from this same period, a very different narrative emerges:

• The AI evaluation company METR ​recently released the results​ of a randomized control trial in which a group of experienced open-source software developers were sorted into two groups, one of which would use AI coding tools to complete a collection of tasks, and one of which would not. As the report summarizes: “Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.”
• Meanwhile, other experienced engineers are beginning to push back on extreme claims about how AI will impact their industry. “Quitting programming as a career right now because of LLMs would be like quitting carpentry as a career thanks to the invention of the table saw,” ​quipped the developer Simon Willison​.
• Tech CEO Nick Khami ​reacted to the claim​ that AI tools will drastically reduce the number of employees required to build a software product as follows: “I feel like I’m being gaslit every time I read this, and I worry it makes folks early in their software development journey feel like it’s a bad time investment.”
• But what about Microsoft replacing all those employees with AI tools? A closer look ​reveals​ that this is not what happened. The company’s actual announcement clarified that cuts were spread across divisions (like gaming) to free up more funds to invest in AI initiatives—not because AI was replacing workers..
• What about the poor CS majors? Later in that same Atlantic article, an alternative explanation is floated. The tech sector has been contracting recently to correct for exuberant spending during the pandemic years. This soft market makes a difference: “enrollment in the computer-science major has historically fluctuated with the job market…[and] prior declines have always rebounded to enrollment levels higher than where they started.” (Personal history note: when I was studying computer science as an undergraduate in the early 2000s, I remember that there was consternation about the plummeting numbers of majors in the wake of the original dot-com bust.)

Here we can find two completely different takes on the same AI issue, depending on what articles you read and what experts you listen to. What should we take away from this confusion? When it comes to AI’s impacts, we don’t yet know anything for sure. But this isn’t stopping everyone from pretending like we do.

My advice, for the moment:

1. Tune out both the most heated and the most dismissive rhetoric.
2. Focus on tangible changes in areas that you care about that really do seem connected to AI—read widely and ask people you trust about what they’re seeing.
3. Beyond that, however, follow AI news with a large grain of salt. All of this is too new for anyone to really understand what they’re saying.

AI is important. But we don’t yet fully know why.

The post No One Knows Anything About AI appeared first on Cal Newport.

View the full article