How to figure out if AI is making you more productive

Using AI in the workplace promises significant productivity gains. And using chatbots may make you feel productive, because it they designed to create engagement from users. But, you need to be more explicit about calculating the costs (and opportunity costs) and tangible benefits to your work. That will help you determine whether the AI juice is worth the LLM squeeze.

Here are three key considerations.

1. Calculate your time spent using AI

When people first started analyzing the downside of smart phones, one of the big data points that got trotted out was how long someone would remain off-task once they picked up their phone. Because apps on your phone are so immersive, once you pick up the phone, it may be 20 minutes before you are back to work on what you were doing before. Based on data like that, phone operating systems started providing users with the amount of time they were spending on their phones and the activities they were engaged in, with the hope that information would guide how people engaged with technology.

LLMs need something similar.

When you sit down to engage with a chatbot or system that will help you build a tool, it creates an engaging conversation that provides you with long responses to your queries and can build tools for you on the fly. When the system is building tools, the models often step through the logic they are using, so you feel like you will miss something if you look away.

As a result, engaging with an AI system can put you in a flow state in which you don’t notice the passage of time. That means you need to track the time you’re spending engaging with AI at work explicitly. That time estimate reflects two costs. First, you have to know whether the value of what you get from the engagement is worth that cost. Second, you should look over your To Do list and determine whether there are other priority items you could have dealt with in the time you spent with AI. The things you could have done with a resource (like time) spent elsewhere is called an opportunity cost, and those opportunity costs often go unnoticed.

2. Evaluate the quality of the output

When you finish engaging with an AI model, you often feel pretty good. For one thing, unless you give the model you’re working with explicit instructions, it tends to butter you up—telling you how insightful and nuanced your thinking is. For another, the model often suggests things you haven’t considered before, so it will take your thinking in a new direction. And flow states in general feel good.

You’re probably used to relying on your feelings as an assessment of whether an experience was good. In the case of AI work, though, you should be more clinical. What was the actual outcome? Did you solve a problem? Did you create an application? Did you make progress on something that you had to complete?

The primary benefit you’re going to get from AI is the product of the work you do with it. That is the only thing you should be weighing against the costs (time, the money you’re spending on your AI platform, etc.). There are many instances in which using AI will truly be worthwhile, but you should be able to document those benefits.

One way to think of this is that your organization probably tracks the productivity of employees in some way in order to determine whether the work they do justifies the HR costs. You should be doing the same thing for your engagement with AI.

3. Are you better off in the long term?

A more subtle issue is that AI ultimately becomes a thought partner. It is scouring the internet for information, synthesizing readings and reports, and providing suggestions based on data. In the moment, those insights may be valuable.

But, those insights also involve cognitive offloading, in which you shift the mental effort of a task from yourself to the AI system. The benefit of doing that cognitive work for yourself is that it often leads to learning and habit creation. This is the same tradeoff that parents and senior business team members face all the time. It is usually faster for a parent to do something for their child or for a more senior person to do a task for one of their direct reports. But, by allowing someone else to do the task for themselves, they build capacities that make them more independent later.

You have to ask yourself whether engaging with the AI model saved you time today only to make your future tasks more time consuming. If you are early in your career and are developing your skills (or are more senior and learning a new area), you might be better off doing a lot of work for yourself in order to build your expertise. You can still engage with an LLM to give you feedback on your work, but bias yourself toward building your own expertise. It is crucial to consider the value of your future self when deciding whether to engage with AI.

View the full article