Understanding Agentic Systems in Generative AI
When we talk about being “agentic,” it might sound complicated, but it’s actually quite simple. In the context of generative AI, being agentic means having the ability to make choices and take actions using the reasoning capabilities of large language models (LLMs). It’s not about magic; it’s about understanding the available tools and deciding which ones to use to respond to a request based on the current context. This context is often represented by memories of previous actions and the results generated so far for this request. LLMs’ reasoning can also be enhanced by retrieving instances of similar scenarios that occurred earlier.
Bridging the Gaps with LLMs
Over time, we have built tools and systems for different purposes. These systems are often disjointed but need to work together to deliver end-to-end value. Traditionally, humans have bridged these gaps, but LLMs have the potential to connect the dots autonomously, enabling full automation. However, this requires careful design and trust in the system’s decision-making capabilities.
Agentic Behavior in Small-Scale Systems
Much like in human society, agentic behavior doesn’t just happen on a large scale but also within smaller scale. For example, instead of building one massive library that’s difficult to search, we can create several smaller, specialized libraries. Depending on the question, we decide which library to consult first to find the needed information. Similarly, we can build multiple indexes specialized for different data sources. An agentic retriever, using LLM reasoning, can decide which index to query based on the request and the specific strengths of each index. Information about each index often becomes part of the input to the LLM’s reasoning process.
Extending to More Complex Systems
We can extend this concept further when building more complex systems, like a chatbot or report generator. You might have retrievers for different unstructured data and other tools, such as a web search tool or tools to query structured or semi-structured databases. LLMs can then decide which tools to use to gather the necessary information to answer a chatbot query or generate content for a report. Even at a simpler level, implementing a mechanism where LLMs just decide if an answer is sufficient or if further information needs to be retrieved is an example of being agentic. So, when people talk about building agentic systems, it doesn’t necessarily imply complexity.
Concrete Example: An Agentic Customer Support System
Imagine a customer support system where a chatbot needs to handle various types of customer inquiries. It can use different tools: a database for order information, a web search tool for general questions, and a specialized knowledge base for technical support. The LLM-based agent dynamically decides which tool to use based on the nature of the inquiry. If a customer asks about their order status, the system queries the order database. If the question is about troubleshooting a product, it consults the technical knowledge base. This decision-making process showcases agentic behavior, as the system autonomously chooses the best course of action using the available tools.
Multi-Agent Systems
This concept can further extend to multi-agent systems. Imagine having multiple chatbots or agents for different knowledge bases, along with existing services like financial analysis, customer management, employee management, and supply management. A supervising agent can decide which tool or agent to use next, based on the available information, the current request, and historical actions. In a multi-agent system, not every subsystem needs to be a full agent; even a deterministic system can act as an ‘agent’ within the broader network.
The Complexity of Agentic Systems
Decisions occur at every level — small ones in agentic tools like retrievers, more complex ones in agent-level systems like chatbots, and even larger ones in multi-agent systems. Anytime you use LLMs to make choices about which tools or agents to use, you are building an agentic system, regardless of its scale.
This approach mirrors our interconnected environment of humans and systems within organizations of all sizes. In some ways, an agentic system can improve results by decomposing tasks into more specialized abilities, enabling self-correction, and addressing errors while interacting with other components. However, as LLMs’ reasoning is applied to more parts, the complexity of these interactions increases, making it harder to refine the system in a deterministic way. Any change can have both positive and negative impacts on different parts of the system.
Challenges and the Need for Monitoring
We need to accept that we cannot build a perfect system, whether small or large — just as humans haven’t been able to do so even for small organizations throughout history. One of the key challenges of agentic systems is managing their complexity. The more autonomous decisions an LLM makes, the more unpredictable the system can become. Changes to one part of the system can lead to unexpected outcomes in another, making it difficult to improve and refine the system in a straightforward way.
To effectively evaluate and evolve the system’s overall performance, we need to monitor it at all levels, from the tool level to individual agents and multi-agent systems. This monitoring ensures that when errors occur, we can trace them back to their origin. It also helps evaluate changes in overall performance and improve the system through trial and error. By consistently monitoring and analyzing these interactions, we can identify areas of improvement and mitigate potential risks.
Conclusion: No Magic Bullet
Therefore, we shouldn’t believe that agents will magically solve our problems and automatically means better performance. They face the same challenges as the non-agentic systems we currently build with generative AI and introduce additional complexities. By understanding and addressing these complexities, we can harness the potential of agentic systems while being mindful of their limitations.
note: The image is generated by openAI.