New VideoDR Benchmark Unites Video Understanding with Open Web Search for Deep Research

Existing multimodal models often struggle with video-based tasks, typically limited to answering questions based solely on content within the video. To address this, researchers from QuantaAlpha, Lanzhou University, Hong Kong University of Science and Technology (Guangzhou), and Peking University have introduced VideoDR, the first Video Deep Research benchmark. This benchmark aims to bridge the gap between video understanding and real-world problem-solving, which frequently involves combining video analysis with web-based evidence gathering.

Person researching museum exhibit details using a tablet and holographic web search.

Traditional Video Question Answering (VideoQA) tasks usually require finding answers directly within a video. However, the VideoDR benchmark proposes a new paradigm for "Deep Research" capabilities, envisioning intelligent video agents that can extend their analysis beyond the video itself. For example, a user might watch a museum video and inquire about the registration number of a recommended exhibit located near one shown in the video. This task necessitates not only understanding the video content (identifying exhibits and their locations) but also searching external sources, such as a museum's official website, for maps, recommended lists, and registration details.

Introducing VideoDR: A New Task Paradigm

VideoDR defines a new task paradigm centered on three core capabilities:

• Multi-frame visual clues: Models must accurately identify continuous key information across multiple video frames for subsequent reasoning.

• Interactive web search: The system should be able to interact within a browser environment to conduct multi-hop, in-depth searches.

• Multi-hop reasoning and verification: The model needs to synthesize video clues and web-based evidence to provide verifiable factual answers.

Abstract visualization of multi-frame visual clues, interactive web search, and multi-hop reasoning.

To ensure the quality of the evaluation, VideoDR utilized strict manual annotation and quality inspection rather than automated generation. The benchmark includes a dual dependency test, removing samples that could be answered by either video observation alone or text search alone, thereby ensuring that models must integrate both capabilities. VideoDR covers six major domains: daily life, economy, science and technology, culture, history, and geography.

Workflow vs. Agentic Approaches

The researchers compared two mainstream paradigms for video deep research:

• Workflow mode: This approach converts video content into structured text clues, which are then used for search and reasoning.

• Agentic mode: In this paradigm, the model directly processes video and search end-to-end, autonomously deciding when to search and when to engage in reasoning.

Comparison of Workflow mode (sequential) and Agentic mode (end-to-end) for AI video research.

The evaluation included both closed-source models such as GPT-5.2, GPT-4o, and Gemini-3-pro-preview, as well as open-source models like Qwen3-Omni-30B-a3b, InternVL3.5-14B, and MiniCPM-V 4.5.

Key Findings and Insights

Evaluation results indicated that Gemini-3-pro-preview and GPT-5.2 performed best, achieving accuracy rates between 69% and 76%, significantly outperforming other models.

The study also revealed that the Agentic mode is not universally superior. While it offers greater flexibility, models in Agentic mode are more susceptible to "goal drift" during long videos or highly complex tasks. The Workflow approach, with its explicit intermediate text acting as "external memory," proved advantageous in preventing models from losing track of initial visual details during extended search chains. Conversely, a drawback of the Agentic mode is that inaccurate initial visual perception, if the video cannot be replayed, can lead to the continuous amplification of incorrect search paths.

Furthermore, long videos presented a significant challenge, exposing limitations in models' ability to maintain long-horizon consistency. Even strong models like Gemini-3 showed performance improvements in Agentic mode when utilizing long contexts, while some open-source models experienced substantial performance declines with longer videos.

Abstract depiction of memory decay and long-range consistency in AI video processing.

The VideoDR benchmark expands the scope of video understanding from confined test sets to the open web. The findings highlight that an "end-to-end" approach is not always optimal, particularly when models face "memory decay" during long-chain searches. Future video agents will require breakthroughs in maintaining long-range consistency of visual clues to effectively handle complex real-world research tasks.