# Deep Research Scoring Criteria

Hello, thank you for participating in the Deep Research scoring process. Before you begin, please read the following scoring criteria. There are five dimensions, each with a maximum score of 100 points. A score below 30 should be given if the standard is severely unmet, 60 points is a passing score, and 90 points indicates that the research has reached the level of an expert team. The following provides a general outline of the scoring criteria.

PS: This standard is primarily intended to help annotators understand the definitions of the dimensions. The specific scores given should be based on the annotator's subjective judgment.
PPS: To avoid bias due to different citation formats in various articles, we have removed all citations. Please do not lower the score of an article because it does not cite its sources; focus on the quality of the article itself.

---

## Comprehensiveness: Does the article cover all the information required by the task?

1.  All necessary materials required for the task have been thoroughly researched and read, and this is reflected in the article.
2.  The information is current and not based solely on outdated materials.
3.  The perspectives are comprehensive, and the information sources are diverse.

**Scoring Rationale:**
Information completeness (reading of necessary + supplementary materials) + Data timeliness (if the task requires it, the article should include the latest information) + Analytical comprehensiveness (multi-angle, multi-source comprehensive analysis).

---

## In-depth Analysis: Does the article deeply analyze the required materials and offer unique and profound insights?

1.  Goes beyond a superficial mention of collected materials, demonstrating an understanding by analyzing the logical relationships between them (e.g., correlations, causal relationships).
2.  Does not merely cite views from reference materials but synthesizes all information to conduct in-depth and professional analysis and discussion.
3.  Proposes new insights, theories, or solutions based on data.

**Scoring Rationale:**
Logical depth (causal relationships + influencing factors) + Strength of argumentation (in-depth analysis supported by examples and data) + Innovativeness (proposing new viewpoints).

---

## Adherence to Instructions: Does the article closely follow the research task and directly address the user's needs?

1.  The entire article revolves around the research task, highlighting core issues, avoiding irrelevant content or digressions, and ensuring the content consistently focuses on key points.
2.  Directly addresses the user's needs or questions at the beginning or end of the article, without evading key points.

**Scoring Rationale:**
Focus on the topic (no digressions or irrelevant content) + Helpfulness of the response (whether it directly addresses the needs).

---

## Readability: Is the language of the article appropriate, its structure clear, and is it easy to understand?

1.  The language used should be appropriate for the research task (e.g., use common language for general questions, and professional language for highly specialized questions).
2.  The structure is clear, with a discernible logical flow between headings, avoiding repetitive statements of the same content.
3.  Uses examples and bold text where appropriate to draw the reader's attention to key content.

**Scoring Rationale:**
Customized language (appropriate for the research task) + Structural clarity (clear logical hierarchy) + Ease of reading (aids reader comprehension).

---

## Overall Evaluation: Comprehensive quality assessment.

1.  Primarily reflects the annotator's overall impression of the article.
2.  Considers the four dimensions above comprehensively, recognizing that some tasks may prioritize comprehensiveness while others may prioritize in-depth analysis.

**Scoring Rationale:**
Overall impression of the article, noting any significant strengths or weaknesses compared to other articles.

---

Thank you for reading the scoring criteria in full. Please begin the scoring process now.
