# How evaluators write

What the corpus behind *The EU Grant Writer's AI Playbook* shows: 70+ real EU evaluation reports — collaborative consortium calls, single-applicant schemes and ERC — reduced to their individual criticisms and strengths, each filed under the criterion it was raised against. Most of the corpus is a private archive, which this file uses only as counts and patterns; a smaller part consists of reports that have been publicly released, and only those are quoted here.

**Use these as patterns, never as predictions.** Criticism counts are not scores: one Excellence objection can sink a proposal while a long list of implementation quibbles barely moves it. Panels differ, and every call is its own case.

## Where criticism lands

Roughly **two in five** criticisms land on **Implementation**, **a little over a third** on **Excellence**, and **about a fifth** on **Impact**. The science and the work plan draw objections at close to the same rate — the work plan, if anything, slightly more. In these reports, the work plan was objected to at least as often as the science.

Many Implementation criticisms are of a checkable kind — a task that is only a title, a missing timeline, person-months never justified — and can be caught before submission.

## The vocabulary panels use

The language of error is essentially absent: words like *wrong*, *incorrect*, *flawed* and *false* do not appear in the criticisms. What fills the space is **insufficiency** — *not sufficiently*, *lack of*, *not clear*, *not fully*, *not adequately*, *unclear*, *only generally*, *not convincingly*. In these reports a claim is almost never faulted for being untrue. It is faulted because the panel could not see enough to judge it: what the tasks are, who does them, in which month, at what cost, and what happens when something slips.

The practical consequence: review a draft for **unverifiability**, not for error. A sentence that asserts without showing is the commonest target.

## Recurring concerns, by criterion (paraphrased)

| Criterion | Recurring concern |
|---|---|
| Excellence | Objectives are not specific or measurable — no indicators, baselines or targets, so nobody can verify success. |
| Excellence | The advance beyond the state of the art is asserted rather than demonstrated; novelty is overstated against prior work. |
| Excellence | The methodology is too thinly described to judge whether it is sound and feasible. |
| Excellence | The ambition is not matched by evidence that it is achievable in the time and with the resources proposed. |
| Impact | Impact is not quantified — KPIs missing, or targets without baselines. |
| Impact | Dissemination is generic — audiences not identified, activities not differentiated by audience. |
| Impact | The exploitation pathway is not concrete; the key exploitable results are not credible. |
| Impact | Impact beyond the project's lifetime — sustainability — is not addressed. |
| Implementation | Tasks are given as a title or a single vague sentence; the section never states what the work is. |
| Implementation | No explicit milestones, and no timeline pinning work packages and deliverables to project months. |
| Implementation | Requested resources are not justified — person-months and cost categories not broken down or explained. |
| Implementation | Risks are not presented as a register in which each risk carries a concrete mitigation. |

Further recurring shapes: something **mentioned but never planned** (a workshop, a board, a stakeholder group that appears in the text but in no task, deliverable or milestone); **inconsistency between text and tables** (effort, commitment or timing that differs between the narrative, the tables and the Gantt chart); **partners with no role** in the core work; horizontal topics (gender, data management, IPR) **addressed generically**; a plan **laid out too broadly** for its structure.

## Self-audit for vague wording

Run this over any draft. Each hit marks a place where a panel may not see enough to judge:

- **Quantity words with no quantity** — *significant, substantial, major, several, a number of, wide range* → give the number, the baseline, the target.
- **Commitment words with no mechanism** — *will ensure, will address, will monitor, as appropriate, as needed* → name the task, the owner, the month, the indicator.
- **Novelty words with no comparison** — *novel, innovative, unique, ground-breaking, first* → name what exists and what this adds, quantitatively where possible.
- **Plans with no place in the work plan** — any activity in the text that has no task, deliverable or milestone.
- **Impact with no audience** — any dissemination or exploitation action without a named target group, indicator and timing.
- **Risks with no mitigation or owner** — every row of the risk register needs both.

## Verbatim examples from publicly released reports

Source: evaluation reports issued under Horizon 2020 and by the European Research Council, as published online by the applicants. These are the only evaluator quotations this assistant may present as quotations. Each is verbatim; attribute it exactly as labelled (scheme and role) and never name or guess the project. Use them to show what a pattern sounds like in a real report — sparingly, one or two in an answer, where they make the point better than a paraphrase. A bracketed ellipsis […] marks words left out of the source here (a detail that would identify the field); keep it when the quotation is reused, and say it is an omission, never part of the original.

**Tasks and work plan**
- ERC Consolidator Grant, reviewer: "In the last tasks experiments are planned but the description of the work planned is somewhat unclear."
- ERC Starting Grant, panel: "the proposed research project is laid out too broadly and a clear structure in the workplan is missing."
- ERC Starting Grant, reviewer: "What is missing is the definition of milestones allowing to assess the feasibility of reaching them in the specified time periods."
- ERC Starting Grant, reviewer: "The timing of the specific activities could have been shown on a high-level Gantt chart."
- H2020 research and innovation action, evaluation summary: "There are some inconsistencies between the GANTT chart and the work package description"
- H2020 research and innovation action, evaluation summary: "Networking workshops for new detector communities are mentioned but not included in the workplan as a task, deliverable or milestone."

**Resources and consistency**
- ERC Consolidator Grant, panel: "The panel felt that the budget was not fully justified under all categories and it has been adjusted under travel, post doctoral and technical staff."
- ERC Advanced Grant, panel: "An adjustment was made to the budget to reflect the discrepancy between the statement of the commitment of the PI in the text and in the table."

**State of the art, novelty and claims**
- H2020 research and innovation action, evaluation summary: "The state of the art is only generally described. The claimed potential advances are not demonstrated in quantitative terms."
- ERC Consolidator Grant, reviewer: "It is not clear that his project really goes beyond the state of the art."
- ERC Consolidator Grant, reviewer: "some important aspects are currently being studied and developed in very similar terms"
- ERC Advanced Grant, reviewer: "What is to some extent missing is the identification of existing challenges which could be solved only using the proposed approach."
- ERC Advanced Grant, reviewer: "it is quite host-centric, while other approaches have been published which could be considered advantageously"
- ERC Starting Grant, reviewer: "It would have been useful to see some recognition of the work done in this field"
- H2020 research and innovation action, evaluation summary: "the proposal does not clearly discuss how the project builds upon and goes beyond previous […] projects in FP6 and FP7."
- ERC Advanced Grant, reviewer: "The extended synopsis argues extremely well the questions 'how?', 'what?', and 'why this PI and this team?', but the first question 'why?' is not perhaps as clearly answered."
- ERC Consolidator Grant, reviewer: "In places there is some over-selling"

**Risk, sustainability, stakeholders**
- H2020 research and innovation action, evaluation summary: "Risk assessment is quite extensive; however, a few mitigation measures are missing (WP20) or are not fully clear."
- H2020 research infrastructure action, evaluation summary: "the strategic vision for the future (after the projects ends) is missing and thus substantially weakening the proposal."
- H2020 research and innovation action, evaluation summary: "The means to involve important and relevant stakeholders is not convincingly presented."
- H2020 research and innovation action, evaluation summary: "Gender dimensions are generically addressed, exact measures are not sufficiently elaborated."

**What praise sounds like** — specific, and tied to structure:
- ERC Advanced Grant, panel: "The proposal gave a good account of the work to be done and a detailed plan of how it would be tackled."
- H2020 research and innovation action, evaluation summary: "The deliverables are clearly related to specific tasks and resources assigned to work packages are in line with their objectives."
- H2020 research and innovation action, evaluation summary: "The KPI-driven methodology is detailed and appropriate for the successful undertaking of the project."
- H2020 research infrastructure action, evaluation summary: "The measures to disseminate project results are good: specific activities and partners are identified for the different dissemination channels."

Anything beyond this list is paraphrase, and is presented as such ("reports in this corpus often say, in effect, that…").
