They Don’t Make ’em Like they Used To

I offer you this toad because it is awesome and you could never make it with AI

We already know that Slack and email don’t make us better workers just by their nature. Used well, they can keep a distributed teams connected and focused. Used poorly, you could spend your whole day distracted and getting nothing meaningful done. Cal Newport’s Deep Work discusses this, along with many others.

This post is about how AI is making that landscape even worse.

Further Fractured Time

Generating AI documents takes time if you are asking for research, and a large output. I am using it to generate requirements for engineering, and also market research reports. I ask it to do something, and I might wait 1 – 15 minutes for the output. While I wait, I check Slack, I get a cup of tea, I start on a different project. I get pulled into something else, I have to join a meeting. So the output completes, and then sits for me to pick it back up later. Sometimes later is the following week. I don’t really know what this is doing to my psyche yet, but it feels like one more way in which work is becoming fractured, and it sometimes feels like we’re using our precious resources to generate all sorts of things that will just sit there and not be read.

Volume

I currently have multiple lengthy AI generated documents that have either been shared with me, or generated by me that are in my backlog to review. I have skimmed them, but they are so dense that I still want to go back and dig in. If I was really consuming them, then I would read every word, click on the links they created as references, comment on them to discuss a point with a colleague. In order to really do that I would probably need at least 2 full days with no distraction except my own need to move and eat. Since we live in a world of Slack, email, and meetings, I don’t know that I will ever be able to do this. Those documents sit in an open tab, or on my to do list. I could maybe get to one per week. This is not much different from before when all the information being out there on the internet or in books that I don’t have time to read.

Well-Being

We should start talking about how this is affecting our well-being. As aforementioned, we are already fractured, burning out, and multi-tasking when studies show that our brains are built to do one thing at a time. Here we are adding one more thing that distracts us, a thing going on in the background, a thing we are supposed to get to but we can’t.

Human in the Loop

The only answers to these problem if you’re starting with a premise that we have to use AI to go faster are: 1. trust the AI that it got everything right, or 2. ask AI to summarize the documents for you. Both of these are flawed approaches because AI is not trustworthy. It gets things mostly right, but there is a good 10 – 20% that it gets wrong. We argue that humans make mistakes too, but also, we specialize as humans so that we trust the information coming from a person who has spent years studying a given topic. We have started to specialize our AI agents. That could work for some things.

It could also lead us down some scary paths if we get to a point where we are just trusting the AI and never reviewing what it outputs. Every human (except the ones trying to profit off of this nonsense) thinks that we still need to be in the loop. And yet I hear more and more people talking about how much of the review we can automate and how little we maybe actually need the humans. This is coming from pressure to go fast and not pressure to make good things. Humans make good things. AI makes approximate things.

Maybe just as with manufacturing, we will start to accept lower and lower quality and just shake our heads and say they don’t make ’em like they used to.”

Day 1

This isn’t actually Day 1. We have been slogging through the depths of the AI revolution for over a year now. The days are long, the stress is real, and the dehumanization is, well, dehumanizing. This is the first day that I have decided to keep this log because I am now at least knee deep in it, and I would like to leave a trail in case I don’t come back out. What I have brought with me is a pencil, my sense of humor, my willingness to try, and my skepticism.

So, what is the landscape here on day 1? We are using these AI tools that some call agents to write requirements so that other agents can read them and produce software. The result right now is A LOT of text that we have to either choose to trust, or to read. I have chosen mostly to read, and every time I do, I find errors. Huge errors? Eggregious errors? Probably not.

All the reading is burning me out and slowly killing my soul. It means that I am interacting with a machine instead of with humans. If this is what Product Management will become, then I don’t think we will attract creative people. Our process has gone from the Agile methodology where we are very close with our teams and work quickly and iteratively, a process that values small chunks of bite sized work and iteration to huge chunks of text, and I assume code, where no human has read every word.

I haven’t yet seen the results. I was supposed to see results quickly. I suspect we’re doing it wrong, but how to convince my fellow travelers? It feels like we’re holding up a lantern in the dark and misunderstanding what we are seeing. I am searching for the light switch. I’ll let you know how that goes.