Unleashing Open-Source DeepResearch: Empowering Web Search Agents

Reimagining AI-Driven Web Search

OpenAI recently unveiled DeepResearch, a sophisticated system capable of surfing the web to distill information and respond to queries derived from these summaries. This innovation marks an impressive leap in enhancing AI capabilities, leaving a significant impression with its first demonstration.

Enhancements in AI Performance

A key highlight discussed in OpenAI's blog is the notable enhancement in results on the General AI Assistants benchmark (GAIA). Here, the system accomplished nearly 67% accuracy on average in one-shot scenarios and reached 47.6% on particularly tricky 'level 3' questions. These questions necessitate multi-step reasoning and effective tool usage.

DeepResearch integrates a Language Learning Model (LLM), which can be selected from OpenAI's arsenal of LLMs, like 4o, o1, or o3. Coupled with this is an internal 'agentic framework' guiding the LLM to utilize tools like web search, organizing tasks methodically.

Venture into Open-Source Reproduction

Motivated by the potential of such a powerful system, a mission was embarked upon to reproduce similar results within an open-source context. This initiative set out to develop and disseminate the necessary frameworks, undertaking this challenge in a tight, 24-hour window.

Understanding Agent Frameworks and Their Importance

Agent frameworks serve as an operational overlay on LLMs, empowering them to execute tasks, such as web browsing or reading documents, in structured sequences. These frameworks significantly amplify LLMs' abilities—transforming them into highly capable systems.

For instance, deploying an agentic framework can enhance performance by up to 60 points. OpenAI's DeepResearch demonstrated remarkable superiority over standalone LLMs when tested against complex, knowledge-intensive benchmarks.

GAIA Benchmark: A Test of Intelligence

Considered a hallmark of complexity, the GAIA benchmark challenges agents to handle intellectually demanding questions, such as identifying fruits in artwork used in historical context menus and tracing their arrangements.

Such questions necessitate:

• Delivering answers in specified formats,
• Utilizing multimodal capabilities,
• Piecing together interdependent information,
• Seamlessly executing high-level plans.

This makes GAIA an ideal litmus test for evaluating agent-based systems.

Constructing an Open-Source Version of DeepResearch

Introducing Code Agents

To surpass traditional AI agent systems, integrating a 'code agent' as suggested by Wang et al. (2024) is pivotal. Expressing actions in code brings several advantages:

• Conciseness: Code typically requires fewer steps compared to JSON, reducing token generation and consequently lowering cost.
• Reusability: It leverages existing libraries.
• Enhanced Benchmark Performance: Coding aligns intuitively with LLM training.

Experiments performed confirmed these benefits, showcasing improved performance on complex benchmarks.

Essential Tool Development

The agent needs the right tools for functionality:

1. Web Browser: Initial tests employed a basic text-based browser; however, expansive performance will demand more robust solutions like Operator.
2. Text Inspector: For efficiently processing various file formats.

The foundational tools, sourced from Microsoft's Magentic-One agent, remain minimally altered, aimed at maximizing performance with streamlined complexity.

Benchmark Achievements and Community Involvement

In a productive 24-hour sprint, the performance on GAIA soared to 55.15% from previous state-of-the-art levels of 46% achieved by Magentic-One. This leap underscores the efficacy of coded actions, as reverting to JSON saw performance dip to 33%.

The journey towards refinement is ongoing, with room for enhancing open tools and exploring superior models. Inviting community collaboration could pave the way for anyone to employ a DeepResearch-like agent locally, customized to personal preferences.

Community Contributions and Next Steps

During this venture, vibrant community versions of DeepResearch sprouted, spearheaded by innovators like dzhng, assafelovic, and others. These alternatives vary in tool integrations, offering fascinating benchmarks for comparing traditional with code-native approaches.

The path forward involves crafting sophisticated GUI agents capable of intuitive interaction via screen views, promising to place advanced tools in everyone’s hands through open-source modalities.

Join the Revolution

Enthusiasts drawn to this vision of democratized AI capabilities are encouraged to contribute. Collaborating on projects like GUI agents can push the boundaries of what's possible in open-source AI, promising thrilling advancements for collective benefit.