When I wrote last week’s letter, I didn’t mention LLM, because I didn’t have to. Then I went to a conference that I love dearly. I love it so much that, when the previous edition ended, I immediately bought a ticket for next year. Later, it pivoted entirely to discussing LLM during the day sessions.

This was a blessing in disguise. On the one hand, there is no way on earth that you will ever make me go to an all-LLM conference on purpose. On the other, I already know most of these people. I’ve even worked with some of them. So no matter what, I’ll be down for the hallway track. I know that won’t be entirely about LLM. I’ll find my people. I’ll also be able to take the temperature of the room. And I’ll do it all in the best food city in North America.

In another dimension, I would have come back throughly LLM-pilled, spending the rest of my life teaching you how to Claude up the joint. I know this will shock all of you, but that did not happen. I have been around the block long enough to both take new developments seriously and also know how to incorporate them into my practice. I literally stare at software like this for a living. I point out the failings of everything in a way that is infuriatingly correct, forever.

Those who’ve been around here for a while already know this, but: LLM is not the totality of your practice. In fact, it is not the practice of most people. Most people have just been handed a tool. Your job is not a tool. Your job has never been a tool.

It was especially fascinating to witness the separation between LLM-as-personal-brand and those who are using it as a tool. Those who use it as a tool were branded “skeptics,” even as they turned around and literally used the thing in ways that clearly delivered value to them. And they all had different ways that they were engaging with it! And that was sort of fascinating.

I use locally-hosted, open-source LLM to synthesize data and transcribe interviews. Nobody seemed to care. At a conference about LLM.

Discernment

If you believe LLM will accelerate the productivity of your whole team, which for the record is not guaranteed, then the nature of your role may change. Paul Ford, the best tech writer of our generation, recently said this:

I’m not really ready to commit to it, but I feel that the roles become more about validation than about production, right? So it’s like a product manager validates that this is a real product. It’s not just like pixels slapped on top of a database, but that it really actually understands the flow of the business or the organization and their needs.

A designer thinks about experience[, components, and] scalability… They are willing to use all the tools necessary to make sure that people can have consistent, reliable experiences when they come and sit down. That is a validation pass, if you are not the one who is moving the pixels around.

People who lack taste are calling this taste. That is incorrect, since taste possesses an aesthetic dimension by definition. The word they’re scrounging for is discernment. Within that is the expertise that has defined these roles for time immemorial.

The tech industry has such a complicated relationship to time. We all want to use new tools to dilate time, to move faster than others. But in order to analyze what an LLM is doing, and incorporate it safely & well into your existing systems, you need to slow down. Everybody in tech hates slowing down, especially in response to using the speed-it-up tool. Unfortunately, I am now the fun ruiner, and you all have to deal with it.

So there is promise here, but also danger. There are never perfect trade-offs. I don’t believe that “making the tool better” will fix them.

Others can get hold of the tool and make whatever, of course. That’s the truly wild thing: that anybody can make software now, with enough time & effort. The barrier to creation has dropped. But the barrier to quality is still there. And quality has real business consequences.

Jank

I recently logged into my payroll provider to pay my assistant, and the form had redesigned. I can deal with redesigns, but this form was clearly buggy in a way that was inconsistent. No tap did the same thing twice. Columns reordered every time I refreshed. You can sniff an LLM-coded thing a mile away. Normally I’d grunt away at this and fire something off and hope it worked, but frankly my payroll system is a financial product, and the well-being of someone very nice & patient with me depends on it.

So I did something very rare for me. I fired off an angry letter. In it I talked about trust. When you’re handling my money and giving it to others I need to be able to sit deeply within that trust. Despite the fact that my payroll system is a disruptoid within a very boring & fraud industry, I still count on them for load-bearing components of my business. Under it I spoke of discernment, of speed.

Anyway, I can smell an LLM rollout from a mile off now. It smells not of slop, but of jank. We expect the interface to do what we tell it to, and now it does not. Jank is when you expect something to possess a certain quality, and now it does not. Also, uh, in this particular case their jobs page is full of LLM roles, so.

Mercifully, the pay-the-assistant page has a “legacy view.” I’ve been using that to pay Tasha since.