AI Guides Creative Solution

Many companies that have started using AI for problem-solving are failing to get the most out of it, according to a new study by Kian Gohar, founder of Geolab, and Jeremy Utley, adjunct professor at Stanford’s School of Engineering. They found that while AI is supposed to help teams innovate, the opposite is often true because most people don’t know how best to use the tool. Teams can achieve much more by treating AI as a fellow brainstorming colleague rather than a one-and-done solution button.

You have recently completed research into how corporate teams use AI for problem-solving. What made you explore this topic?

Kian Gohar: We always try to help companies and organizations find the most effective solutions to their problems. When GenAI became widely available, it dawned on us that this could be a very interesting technical tool for facilitating better ideas. So, we designed this research initiative to examine the role of GenAI in supporting ideation for teams trying to solve a problem.

The results were not quite what you expected. Can you explain?

Jeremy Utley: We expected that, with GenAI, teams would be more innovative, they would generate more ideas, and those ideas would be more diverse. Our question was, how much more innovative would they be? 10 times? 100 times? So, we were amazed when those teams using AI performed worse. It just didn’t amplify innovation outcomes like we thought it would.

When you look back at other research, it makes sense. What afflicts problem solvers is this kind of cognitive bias that Abraham and Edith Luchins called the Einstellung effect back in the 1940s. The crux of it is that, as human beings, we tend to settle for good enough as quickly as possible. There’s a reason that creativity often doesn’t happen in the conference room, and it’s because there’s a deep human longing to answer a question and move on. However, we know that a volume of solutions is what yields breakthroughs.

Whereas one would think that AI would liberate us from the shackles of this human cognitive bias, we often find that it only amplifies the underlying cognitive bias. A human team might take 30 minutes to get to a point where they have brainstormed a good idea and then maybe refine and develop it further. The AI-assisted teams, however, considered they had a pretty good idea within about 5 minutes, and then they would settle with that good enough idea. They didn’t then leverage the AI to amplify the possibilities. It simply amplified their underlying bias.

You concluded that corporate teams could get more out of AI by treating it like a conversation partner rather than a search engine. What advice would you give to people or businesses to action this?

Jeremy Utley: We developed a model we’ve coined ‘FIXIT’. The ‘F’ stands for ‘focused’. So, set a focused problem—be precise instead of abstract. Don’t try to boil the ocean. 

The first ‘I’ stands for ‘individual.’ We need to safeguard human creativity by striving for individual ideation first. We should see our brains as our private large language model instead of treating AI as an oracle that we outsource to. Take the responsibility to ideate individually first; you will get much better input and, as a result, much better output. 

The ‘X’ stands for ‘context’. Provide context to train the AI. You can even use it as a thought partner if you don’t know what context to give AI. So, you could say, ‘Would you ask me four or five questions about this problem so that you have sufficient context to help me solve it?’ That’s something you would never ask Google, and it illustrates how AI is a radically different orientation.

The second ‘I’ stands for ‘interactive.’ Have interactive conversations with AI as a thought partner. The regenerate button is one of the most important elements of a GenAI solution. It’s essentially asking it to try again.We suggest you hit that button three or four times, anytime you ask a question, just to survey the landscape of responses. That’s because it’s nondeterministic and, unlike Google, it won’t give you the same results every single time. They might be similar, or they might be radically different. 

We aren’t used to interacting iteratively with technology. Part of the reason is that technology has trained us wrong. We may well program technology, but it is also programming us—it’s programmed us to expect that the first result will be the best result.

Finally, the ‘T’ stands for ‘team’. Team incubation facilitates decision-making.

Kian Gohar: The user interface looks like a search engine when you go into a generative AI solution. We have been conditioned for decades to interact with search in a particular way. You type something in, and you ask for the perfect answer. It’s what we call the tyranny of the search.

In our research, we found a few teams that consistently got better responses and ideas from AI compared to all the other AI-assisted teams. It became clear to us that their workflow was very different from the others. They gave their query context and had a back-and-forth conversation with it. That made us realize that you can get significantly better ideas if you approach AI differently. 

You say that businesses need worse ideas to succeed. Can you explain what you mean by this?

Jeremy Utley: It’s a bit of a provocative statement. We mean that it is important to increase the variation in your thinking. At a recent music awards ceremony, Taylor Swift said: “I really want young people to know that it’s the hundreds of thousands of dumb ideas that I’ve had that have led me to my good ideas.” When you’re willing to have bad ideas, you increase the variance of thought and open the door to many more possibilities. And this is where good ideas often stem from. 

Firms like Microsoft are offering AI-driven ‘copilots’ for their software. Do you expect us to see more of this, and is this the right approach?

Kian Gohar: AI is in an intermediate phase right now. In the next two years, we will stop using standalone generative AI platforms and embed them into every software platform we use.

It’s like the early days of the Internet – it took us a decade or so to get good at it and to create more intuitive web browsers. We will see the same thing happen with AI. The copilot technologies that are just entering the market are an intermediary step. It will be very interesting to see how people adopt these technologies inside their organizations, and we will focus some of our upcoming research on this subject. 

Kian Gohar: Getting the most from AI has a huge generational component. If you’re in your 20s or 30s, you are likely willing to use these technologies daily. However, those in their 40s use them far less often, and those in their 50s and 60s fear using these sorts of technologies. It’s a problem. You can’t just write off half of your workforce.

Jeremy Utley: Organizations miss out if older employees don’t feel fluent. We’re seeing this generational gap emerge, and it’s a troubling sign because the outputs you get from AI are only as good as the inputs you can feed it. 

How can organizations meet this generation gap?

Jeremy Utley: Education is important here. It is essential to train all employees on AI to benefit from these productivity tools. Kian and I are working on a coaching tool that provides users with a daily AI drill. It gives some suggested prompts for typing into a generative AI solution. Users get small chunks of experience that add up.

To use a personal example, I was talking with my elderly grandma. She had a deep, important question about her personal life, so we used GenAI to discuss it. She was blown away. Over the next week, she asked me, “Do you think we could use GenAI to help me find a substitute for cream of mushroom soup in the green bean casserole recipe?” and “Do you think we could use GenAI to help us come up with family photo options?” It turns out the answer to pretty much any question is ‘yes.’

Kian Gohar: Our work is focused on behavior change and transformation at an organizational team level. While you can talk about something theoretically until you’re blue in the face, people won’t change their behaviors. People know they shouldn’t be eating sugar or drinking diet soda, but until they change their behavior, they won’t change their lives. This is the point of our drill coach. We can talk about why it’s important, but you won’t get good at it unless you start practicing it daily. It lowers the bar, and then, over time, people will develop the muscle memory to use AI consistently.