Generative AI is everywhere today. For most of us, it's a chatbot that helps us write emails, search for quick answers, draft essays, or even create poems to apologize for a forgotten birthday!
Most interactions with large language models (LLMs), like ChatGPT, involve asking simple questions—a process known as zero-shot prompting. The chatbot gives fast, intuitive responses based on its training data. Psychologist Daniel Kahneman, in his book "Thinking, Fast and Slow"¹, describes this rapid and intuitive process as "System 1" thinking. These answers are often good enough but can be wrong or misleading. However, by using better prompts—like asking the model to consider different options—we can get more accurate results.
Newer models, like ChatGPT's "o1," are designed to mimic Kahneman's "System 2" thinking, which is slower and more deliberate. They focus on providing thoughtful, accurate answers rather than quick ones.
How to improve chatbot responses
There are several systematic ways to improve the quality of chatbot responses when using an LLM:
- Better prompting: Carefully crafting or refining prompts (the instructions and context given to the model) can improve the quality of responses. This is something seen in many professional fields, including marketing, data analysis, and education. Scrum masters, for instance, can benefit from learning how specific prompts can enhance team communication and problem-solving
- Fine-tuning: By training models on specific domains, we can make them faster and more accurate for specialized tasks. For example, a model like BioBERT, trained on medical literature, performs better on healthcare-related questions.
- RAG architecture²: Retrieval-augmented generation (RAG) combines LLMs with curated data sources for more precise answers. For instance, Stevie Ai, an AI Agile Consultant uses selected articles and books from well-known scrum experts to answer questions. By linking to focused repositories, RAG systems ensure contextually accurate responses.
The next step: agentic AI
So far, we've focused on improving how an AI chatbot answers questions. But what if our bot goes beyond answering questions and starts taking action? This is the idea behind agentic AI, when a bot becomes more like an agent with a goal. It uses the reasoning power of AI to make decisions and take actions to achieve that goal.
For example, instead of just answering questions, an AI agent could:
- Solve math problems
- Search the web for information
- Send emails
- Update databases
- Handle complex workflows
This capability mirrors the business application layers of today's software. Microsoft CEO Satya Nadella envisions³ these AI "agents" becoming central to enterprise SaaS products, transforming how users interact with software and how businesses operate.
These agents could transform industries. Take healthcare: a kidney transplant coordinator, often a busy nurse practitioner, must gather information, coordinate with specialists, and manage scheduling. An AI agent could automate much of this, tracking results and presenting a clear dashboard to help the transplant team decide whether to proceed with surgery. This would save time and allow the nurse practitioner to focus on patient care.
AI in software development
When ChatGPT first launched, it wasn't perfect for coding. It often produced errors that required debugging. In these cases, the human user acted as the agent, correcting and guiding the AI to create functional code. Now, imagine an AI agent that could autonomously write, debug, and optimize code, following coding standards and security protocols. This is what platforms like Factory.ai aim to achieve—helping organizations automate software development and optimize processes through AI-human collaboration.
Iterate faster in product development
Even today, AI can speed up how we develop and test ideas. For example, a product owner's role is to ensure the team builds the right product by testing ideas with customers early and often. Generative AI can help product owners and product developers by allowing them to quickly prototype solutions before involving the team. With basic coding skills or a teammate's help, they can use AI to experiment and refine ideas, speeding up iterations and improving outcomes.
While tools like Copilot speed up coding, generative AI can help product owners move faster, too. This isn't about having ChatGPT write user stories— that's better done collaboratively with the team. Instead, product owners can use AI to rapidly test ideas. With basic Python skills or help from a teammate, they can quickly prototype solutions before involving the team, enabling faster iterations with customers to refine the product.
We began our use of generative AI like ChatGPT by using it for fast, intuitive responses, progressed to more deliberate reasoning, and now we are at a stage that includes reason and action. These advancements are redefining how we use generative AI and even building software applications. This is definitely accelerating innovation and business processes. There's so much to learn. Are you ready? Is your organization ready? What's your next experiment?
Author’s note:
As this article was being prepared for publication, OpenAI announced the release of Operator, an exciting advancement in the field of AI agents. As a ChatGPT user, you can simply ask Operator to go to the web and perform tasks autonomously using its own browser. This development represents a significant leap forward in AI capabilities, aligning closely with the ideas explored here and highlighting the rapid progress of AI technologies. For more details, see OpenAI's release notes: Introducing Operator.
Reference:
- Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.
- What is retrieval-augmented generation, and what does it do for generative AI? –GitHub Blog.
- Satya Nadella | BG2 w/ Bill Gurley & Brad Gerstner –Watch on YouTube at 46:40.
--
Brush up on your AI skills and know-how
The author of this article, Manoj Vadakkan, is an AI and agile subject matter expert who contributed his expertise to the development of these two Scrum Alliance AI courses:
Scrum Alliance course developers are thrilled to work with subject matter experts to bring our members effective, interactive, engaging lessons in today's most in-demand professional skills. Take both of these courses to learn how to improve team communication, manage timelines, predict risks, and bring the right products to your customers. Enroll today!