We adopted a multi-agent system, where each agent is responsible for a specific task. Working in a sequential manner, the agents build on the outputs of the previous stages, resulting in more accurate and coherent documentation while reducing hallucinations. By focusing on specific domains and tasks, each agent contributes to the overall accuracy, following a chain–of–thought process.
Multi-Agent System Architecture
In our implementation we used several state-of-art Microsoft tools including Microsoft Semantic Kernel and Microsoft Autogen, as well as GPT-4-0125-preview for powering all agents.
The multi-agent conversation, as mentioned, follows a sequential pattern, which means the agents perform a sequence of chats, chained together by a carryover mechanism to complete a task. The carryover, which is the output of the previous chat, is passed to the next chat as the context information so that subsequent agents have the knowledge from previous chats.
The baseline response generated by the documentation generation agent is potentially prone to hallucination. Therefore, we use the LLM self-check mechanism to review and improve the initial documentation, which consists of four steps:
Generate function relationship data
This Code Context Agent creates a graph representation of the entire repository, mapping out the relationships between function calls. It then generates comprehensive information about the code by using both the actual source code and the relationship data from the graph related to that source code.
Generate a baseline documentation
The Documentation Generation agent generates documentation for the code, taking into account the context explanation returned from the code context agent.
Review the documentation
The Review Agent checks the documentation and generates a review report, including comments and improvement suggestions.
Generate improved documentation
The Revise Agent receives the review report, modifies the documentation according to the suggestions, and finally returns an improved documentation to the agent manager.