(Photo by Simon Kadula on Unsplash)

(This is a follow-on to my previous post, on developers and bots creating flawed code. Here I take a deeper look into the wider impact of genAI on the entire application development and delivery process.)

When I talk to people about business models, I encourage them to skip over "will this work?" and focus on "under what circumstances could this work?" What looks like language-lawyering on my part actually opens up the discussion – you start with the premise that the thing already exists, and you work backwards to understand the world around it. From there it gets easier to spot threats, opportunities, and alternatives.

I approach AI-based code generation (or vibe coding, if you prefer) with a similar mindset: if we start with the premise that code generation is here to stay, how do we describe the world in which it works well?

A recent conversation with Mike Loukides sharpened some of my ideas on this topic and gave me some new ones to think about. The gist is that the professional app dev realm will have to double down on some existing practices, plus learn some new tricks.

Soft skills are back

Earlier in my career, employers sought developers with so-called "soft skills" – mostly, the ability to communicate clearly in written and spoken form. At least, that's what they said. Hiring managers often let that slide. But genAI may make soft skills a firmer requirement.

To make the most of LLMs, developers need more than the ability to review the code that comes back from the bot. They need to explain, clearly and in detail, what they want it to build. They also need the patience to repeatedly ask it to make a fix, plus a fine-tuned sense of when to do the job by hand because the bot keeps getting lost.

The clear speech required to guide a code bot is often lacking in social graces. It is firm and direct in a way that would breach social norms. Developers will therefore need another soft skill, which is the ability to shift gears – speaking with their fellow humans in an office-appropriate manner while issuing sharp orders to a bot. Those who aren't able to context-switch may find themselves in hot water.

We need real specs

The notion of a software project spec has always been hazy. It's sometimes just a loose idea jotted down in a ticket. A team of human developers can always fall back to ad-hoc discussion and shared understanding to compensate for lack of formal documentation. (I'm not saying that this is a good idea, just that it happens.) And with an in-office team, especially, it's easy to have conversations that don't get documented in Slack or other chat systems. You can toss an idea over the wall because the other people in the room know what you mean. Just don't ask them three years from now when it's time to rebuild that system.

With LLMs, your detailed prompt becomes the spec. Saving that prompt will be as important as saving the source code. When the time comes to update that code, ideally you'd run that same document through a newer version of the bot – one that had picked up on newer versions of your code dependencies.

This situation underscores an oft-neglected aspect of software development: understanding what, specifically, we are trying to build. In order to write a spec that's detailed enough for an LLM to handle, you have to know what you want. You can't hand the bot something vague and hope for the best. (Too many dev teams experience this already, and it rarely ends well.) Tracking changes to the spec, through commits to source code control, can also shed light on a team that's going in circles. Which may in turn point to a leader or customer who can't decide what they want.

Other roles get more involved

Code generation doesn't automate the entire application development process; it automates writing code. And as I noted earlier this year:

[A] typical software project involves so much more than code. There are test suites and best practices and different stages of migration. There are product people and QA teams and infrastructure specialists, the combined efforts of which create a system of checks and balances to keep everything on the rails.

Mike pointed out that QA becomes more important when there's generated code in an app. He's right. And his observation extends to all of the other roles I mentioned. If they collectively protect against developer error (or worse, a developer gone rogue) then they'll have even more work to do when the developer-bot churns out far more code in the same time frame as its human counterpart.

The QA, product, and ops teams aren't just dealing with a greater volume of code; they're likely handling weirder code. If bots can cook up interesting moves in poker or chess or Go, you'd best believe they'll do the same when generating code. And since fixing problems in generated code will likely become tougher to manage, spotting bugs early will become even more important.

Learning to let go

Consider the apps on your phone, or the SaaS apps you use every day. They're a black box to you. You don't even think about their inner machinery so long as they work as expected.

To a developer, LLM-generated code feels like a black box. Even if they have the necessary experience to review it, the sheer volume of a bot's output can overwhelm their ability to do so. The catch is that those developers are on the hook for the code it generated, which can worry some people into limiting a bot's contribution to a codebase.

As LLMs improve, and as the code becomes more reliable, we'll be able to treat their outputs as SaaS apps. And just like SaaS apps, they will occasionally goof, and we will find ways to limit the damage. That leads to the next point:

Containment

Keeping generated code in its own area will help with letting go of the proverbial wheel.

Developer teams already see apps as a set of connected components. Sometimes it's as simple as "front-end (UI) and back-end (business logic)." The front and back parts of a sufficiently large app further decompose into components that are handled by separate teams, and communication between them takes place through API calls. The next logical step is to hand certain components off to the LLM "team."

This containment won't eliminate AI-generated bugs, but it will make it easier to focus monitoring, testing, and troubleshooting efforts. Companies could strap a lot more monitors onto the vibed-up portions of the app – especially those that take on more responsibility – and take a different approach to debugging when a problem arises in those areas. In some cases a "fix" will really be "don't waste time on troubleshooting; rerun the spec through the bot." So long as you trust your test suite, why not?

Defining these boundaries also clarifies where generated code is welcome. Perhaps Component A is full of sensitive business logic that's better written and maintained by someone with deep domain expertise, while Component B has a higher tolerance for error.

Smaller projects can serve as testbeds for this idea. Case in point: back when I could still call myself a software developer, I specialized in backend systems and machine-to-machine interaction. Did I perform occasional front-end web work? Sure. Did I do great front-end web work? Far from it. Did I ever improve? Not at all. Since then I've mitigated the problem by simply staying away from front-end work. If something I build needs a UI, I partner with the talented web developers I've met over the years.

What about those one-off, low-stakes situations in which the UI is not worth those web developers' time? I currently cobble something together with my limited web dev knowledge; maybe someday, as the code-generation LLMs improve, I'll entrust that work to a bot. Even today's code-generation bots are one thousand times the JavaScript pro I'll ever be. Getting them to write those parts would speed up my time to release and the product's end-users will thank me for a workable UI.

Will it get better or worse?

Going back to my conversation with Mike Loukides, he ended on a question:

When we look at LLM code, over time, will the situation get better or worse?

My take?

Yes.

For some in the industry, it definitely gets better. Developers who write code for the sake of solving a problem (not for the love of writing code) are already shifting into more of a team lead or manager role, where they can focus on higher-order matters and not get hung up on line-by-line code minutiae. We'll see more of that shift over time. Anyone who can orchestrate bots into building large, complicated systems while still ensuring application reliability will become the mythical "10x developer" we've heard so much about. And I mean developers who are actually ten times as productive; not those who simply boast that they are.

The flip-side of this coin is that work intensifies for everyone on an app dev team, not just the developers. There will be more boundaries to erect and defend. Expect tighter and more frequent deadlines, too, as company leadership will push people to keep up with the machines' output.

The future is not looking so rosy for entry-level developers. Nor for the more senior people who simply love to write code. The machines aren't great at your job yet, but they're already pretty good. And since code generation has proven to be a rare successful use case for genAI, expect companies to pile more resources into improving it. You can hope for a widespread genAI crash; but more than likely, you'll either have to accept your role as a bot-manager or find a different job (and write code for hobby projects in your spare time).

End-users get a mixed bag. The Dot-Com rise of free developer tools, followed a few years later by the arrival of low-cost cloud computing, gave us a long list of apps and services. Throwing code-generation bots into the mix will grow that list even more, taking us even deeper into the long tail of niche solutions. Some of these will be great and some will be terrible. (Even professionally-managed apps can be terrible, and some hobbyist apps can be wonderful.) We won't be able to tell the good from the bad until we encounter a problem, though. So we should expect more security flaws and other app failures that will frustrate us and leak our personal data.

All in all, the arrival of code-generation tools has created new opportunities for software developers and their end-users. With all of this new code – amateur and professional, flawed and stable, popular and niche – I expect we'll see a larger, faster-moving app ecosystem compared to what we had just a few years ago.