Agentic T-STROBE: a front-shifted STROBE paradigm for observational research

Chunxin Bo; Ruimin Li; Jinghua Zhao; Shiqing Feng; Xiaokang Ji; Fuzhong Xue; Chunxin Bo; Ruimin Li; Jinghua Zhao; Shiqing Feng; Xiaokang Ji; Fuzhong Xue

doi:10.48130/stati-0026-0014

The reproducibility crisis is a structural consequence of research complexity surpassing human capacity—a limitation of the 'artisanal science' paradigm. Current guidelines like STROBE function as retrospective post hoc checklists. We introduce Agentic T-STROBE, a framework that reframes methodological completeness and procedural rigor as structural constraints intrinsic to the research workflow, rather than criteria applied in post hoc evaluation. It employs (i) front-shifting STROBE to pre-emptive design constraints; (ii) a large language model (LLM)-driven dual-agent system programmatically executing workflows; and (iii) the Predictability–Computability–Stability (PCS) framework as an organizing evaluation lens. Agent-generated protocols (instantiated as AgenMed_CSS) demonstrated methodological completeness superior to 300 peer-reviewed studies (mean percent of maximum possible (POMP) score: 98.3% vs 68.3%; p < 0.001). This advantage was consistently observed across four LLM backends (mean POMP [95% CI]: 99.2 (98.5–99.8), 98.7 (98.0–99.3), 99.4 (99.0–99.8), and 97.2 (96.0–98.3)), suggesting that the observed methodological completeness is more contingent on constraint architecture than on the capabilities of the underlying model. The statistical execution agent achieved > 92% fidelity. Conversely, unconstrained human-AI collaboration yielded inferior quality (mean POMP: 63.1 [56.9–69.2]). Agentic T-STROBE establishes a prospective, auditable oversight framework for methodological completeness and protocol transparency, without substituting scientific judgment.

HTML

Introduction

The discourse regarding the reproducibility crisis has dominated metascientific inquiry for more than a decade, driven by the alarming frequency with which published findings fail independent verification^[1−4]. While conventional diagnoses often focus on individual researcher negligence, p-hacking, or the perverse incentives of the publish-or-perish culture, these explanations remain superficial. A deeper, structural pathology exists: the researcher-dependent workflow^[5−7] that currently underpins scientific production has reached its cognitive limit. In this 'artisanal' mode, complex, multistage methodological tasks are contingent upon the tacit knowledge, cognitive stability, and memory of individual researchers^[8]. As the complexity of modern scientific inquiry grows exponentially, this 'human-centric variability' has solidified as the fundamental bottleneck restricting research reliability and cumulativity^[8].

To mitigate this variability, the academic community has invested heavily in standardization, most notably through the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement^[9]. STROBE and its extensions provide a comprehensive checklist covering critical aspects of research design, execution, and analysis, intended to enhance transparency and completeness. While STROBE represents a milestone in methodological dissemination^[10,11], its utility is inherently limited by its nature as a descriptive, post hoc reporting guideline^[12,13]. It delineates how to describe research clearly, not how to execute it correctly. It functions as a map rather than a navigation system; a blueprint rather than a construction robot. Consequently, a significant practice-norm gap persists, in which adherence to guidelines remains voluntary and inconsistent.

The advent of artificial intelligence has introduced the concept of the Large Language Model (LLM) as a research 'co-pilot'^[14−16]. This prevailing model typically relies on unstructured, free-form interactions between researchers and general-purpose models to accelerate specific tasks such as literature review^[17], code generation^[18], or manuscript refinement^[19]. However, this mode fails to address the core issue of methodological reliability. In the absence of structured methodological frameworks and rigid constraint mechanisms, unconstrained human–AI interaction cannot guarantee methodological correctness^[20−22]. Indeed, by introducing model hallucinations^[23−25], exacerbating user knowledge blind spots, and adding the stochasticity of conversational interaction, this model risks amplifying rather than reducing human-centric variability.

The emergence of LLM-based agentic systems—capable of autonomous planning, tool invocation, multi-step reasoning, and self-correction—represents a qualitative advance beyond the co-pilot paradigm^[26,27]. In the biomedical domain, agentic frameworks have demonstrated early promise across a range of tasks: BioMedAgent autonomously chains bioinformatics tools into executable workflows for cross-omics analysis and machine-learning modelling^[27]; agentic systems in radiology automate report generation and protocol optimization^[28]; and multi-agent LLMs have been applied to collaborative hypothesis generation in drug discovery^[29]. Comprehensive reviews synthesizing over 60 emerging agentic systems consistently identify reproducibility and methodological grounding as the central unresolved challenges^[26]. Critically, however, existing agentic systems share a common limitation: they are designed for task execution within established analytical pipelines, not for a priori methodological constraint enforcement in the research design phase. None of the existing systems encodes domain-specific reporting guidelines (such as STROBE) as executable design rules, nor do they generate a complete, auditable computational lineage spanning from research question formulation to statistical analysis execution. The gap between automated task completion and guaranteed methodological rigor in observational epidemiology therefore remains unaddressed.

We argue that the necessary evolution in scientific rigor demands a paradigm shift from descriptive guidelines to an Epistemological Operating System. Such a system must transform methodological best practices from passive text into active, executable contracts. To this end, we propose and validate Agentic T-STROBE (Agentic Target-STROBE Emulation), instantiated here as AgenMed_CSS for cross-sectional studies (CSS). To our knowledge, this system represents the first agentic framework specifically designed to enforce a priori methodological rigor in observational research design—distinguished from existing systems by its integration of front-shifted reporting guidelines as executable constraints, its dual-agent planning-execution architecture, and its generation of a complete, auditable computational lineage. It is not merely a tool but an engineered paradigm designed to make methodological rigor an a priori system property.

The Agentic T-STROBE paradigm is founded on three synergistic principles:

1. Agentic Target-STROBE Emulation (Agentic T-STROBE): This principle involves front-shifting the STROBE checklist. Instead of serving as a retrospective check on a finished manuscript, STROBE items are reconstructed into a set of mandatory, executable design rules and deliverables—specifically, the research protocol, statistical analysis plan (SAP), and simulation tables—that govern the research lifecycle from its inception.

2. Agentic Execution: We construct a dual-agent system that programs these rules and complex methodological decision trees into a researcher-supervised automated workflow. Within this workflow, structured user input and protocol review precede automated statistical analysis, and complete auditable logs are generated throughout, supporting high-fidelity implementation of methodological requirements.

3. PCS (Predictability–Computability–Stability) as a metric of veracity: Drawing upon the theoretical framework of veridical data science^[30], we integrate PCS as an organizing framework for process-level reliability assessment. This facilitates a shift from subjective peer review to quantitative quality assurance.

Through comparative experimentation, we evaluate whether protocol completeness and execution fidelity are primarily supported by the structured workflow rather than by a specific LLM backend or unconstrained human-AI collaboration. The results indicate that methodological completeness functions as a property of the workflow that can be specified, audited, and reproduced across different models and users, rather than as an attribute contingent on individual memory or ad hoc judgment.

Materials and methods

AgenMed_CSS system architecture

Overall design

The AgenMed_CSS system is architected as an end-to-end 'planning–execution' dual-agent cognitive workflow. This framework is explicitly designed to decompose the complexity of scientific inquiry into two distinct, specialized operational phases: a high-level strategic planning phase, governed by the Planning Design Agent, which is responsible for the formulation of rigorous research protocols; and a low-level implementation phase, governed by the Execution Analysis Agent, which is tasked with precise code generation and local execution.

Planning design agent
To anchor the LLM's inferential reasoning within strict methodological boundaries—thereby neutralizing the risk of hallucination—we encoded the front-shifted STROBE workflow into a granular decision topology. This knowledge graph serves as a deterministic scaffold, strictly confining the LLM's execution trajectory to validated standard operating procedures throughout the entire research lifecycle (see Supplementary File 1, Material S1).

We developed AgenMed_CSS as a multi-module collaborative system comprising modular prompt chains (see Video S1 for the workflow structure and node-level prompt configuration) built on the n8n agent-workflow platform (https://github.com/n8n-io/n8n). In this agent-workflow, the decision tree (see Supplementary File 1, Material S1) was decomposed into 52 nodes, each with a precisely specified LLM role, task, methodological guidance, and standardized JSON outputs stored in PostgreSQL for real-time retrieval. This architecture enables high-fidelity, end-to-end cross-sectional study design and analysis while ensuring full traceability, auditability, and transparency.

For a given cross-sectional study design and analysis project (e.g., a population-based cross-sectional survey of bone mineral density among adults aged ≥ 65 years in Shandong Province, China), the workflow begins with researcher input rather than autonomous question generation. Users first select an appropriate question-structuring framework according to their research objective: PECO (population, exposure, comparison, outcome) for association-oriented questions; CoCoPop (condition, context, population) for prevalence estimation; and SPIDER (sample, phenomenon of interest, design, evaluation, research type) for qualitative or mixed-methods research. A combined PECO–CoCoPop structure is also available for studies that pursue both objectives. Users then complete the corresponding structured fields, and these researcher-specified inputs anchor the planning design agent's subsequent protocol generation (see Video S2 for a demonstration of protocol generation and statistical analysis, and Supplementary File 1, Material S4 for the research protocol).

The generated protocol is organized into nine chapters: Title and abstract, Background and objectives, Study design and setting; Participants; Variable definitions and data sources/measurement; Bias control and sample size; Statistical methods; Ethical statement and funding; Research results publication and dissemination plan. Within these chapters, the 22 STROBE_CSS items are expanded into 32 scorable protocol elements (see Supplementary File 1, Material S2 and S8). Specifically, the above full-process decision tree (see Supplementary File 1, Material S1) encompasses the entire research lifecycle—including research question framework selection, definition of target/source populations and sampling frames, sampling design and sample size calculation, variable measurement and tool selection, bias control, missing data handling, association/regression modeling, and results reporting. Furthermore, predefined statistical decision tree rules, constructed based on the principles of statistical description and inference for cross-sectional studies^[31], are embedded within the relevant modules (see Supplementary File 1, Material S5). These rules enforce mandatory specifications, such as assumptions and methods for sample size calculation, strategies for missing data handling, schemes for complex survey weight application, and protocols for model specification and diagnosis. The decision tree serves as the default methodological pathway, while justified deviations required by the study question, data structure, sampling design, or model assumptions are permitted with documented rationale and audit-trail preservation.

By encoding methodological constraints into reusable decision rules, this structured design aims to maximize the reduction of methodological fluctuations arising from natural language ambiguity, thereby ensuring a predictable baseline for the generated protocol regarding terminology usage, content coverage, and methodological consistency. Importantly, this constraint structure does not remove the researcher from the design process. The generated protocol functions as a reviewable artifact through which investigators can refine population definitions, variable specifications, covariate assumptions, and the SAP before statistical execution begins.

Execution analysis agent
Operating within a user-localized R/Python runtime environment, this agent executes a closed-loop 'Parse-Generate-Execute-Repair' workflow by invoking external large language models (LLMs) via real-time API calls to parse protocols and generate code.

It accepts the researcher-reviewed statistical analysis plan (SAP), embedded within the protocol generated by the planning design agent, and the local dataset as primary inputs. First, the system parses the structured constraints of the SAP to generate hierarchical sub-task prompts. Subsequently, R code is generated based on these prompts and metadata (e.g., variable names, types, and missing data profiles). Crucially, to mitigate randomness in code generation and ensure consistency with the SAP, the code generation phase employs a low temperature setting (temperature = 0.1) for the LLMs (e.g., Claude-Sonnet-4, ChatGPT-5)^[32,33].

Prior to execution, the system enforces a pre-check gateway defined by protocol-specified methodological constraints, including any approved deviations recorded in the SAP. Code passing this pre-check is executed within the local environment. In the event of execution failure, error logs are captured and categorized, triggering an automated repair loop (with a maximum of 10 iterations). The system retries using a minimal modification strategy until successful execution is achieved (see Supplementary File 1, Material S6). The resulting code, tables, figures, repair traces, and execution logs are then available for investigator review before the statistical report is finalized.

Data security and provenance mechanism
The entire workflow of the execution analysis agent strictly adheres to a 'Data Does Not Leave the Domain' principle. External LLM APIs receive only desensitized metadata (e.g., variable lists) and error logs; raw data remains permanently resident within the local environment. All artifacts generated during the process—including multi-level prompts, model versions and parameters, generated code scripts, execution results (tables and figures), accepted methodological deviations, and comprehensive system logs—are automatically archived into independent working directories named according to chapter numbering. This creates a complete, immutable chain of evidence suitable for peer review and future reproduction (see Supplementary File 1, Material S7).

PCS framework for veridicality assessment
To evaluate the reliability of the research outputs, this study adopts the PCS (predictability, computability, stability) framework from veridical data science as its theoretical foundation.

Predictability evaluation
Predictability is operationalized as a reality check in three dimensions.

First, regarding the predictability evaluation of the intelligent design protocol produced by the planning design agent, we initially refined the STROBE_CSS Statement into a 32-item quantifiable scoring rubric^[34,35] (see Supplementary File 1, Material S8). Using this rubric as the comparative benchmark, we uploaded both the rubric and the target text—either the original published articles or the intelligent design protocol produced for a specific query—to the LLM (e.g., Gemini 2.5 Pro). Guided by the established prompt engineering (see Supplementary File 1, Material S9), the system conducted a quantitative performance evaluation of the intelligent design protocols derived from original published articles (or those generated de novo for specific queries).

We emphasize that the STROBE_CSS-based POMP score was designed to quantify checklist-based methodological completeness and the correctness of predefined methodological steps, not to assess intrinsic scientific validity. The 32-item rubric evaluates whether key design and analytical elements are explicitly addressed and methodologically appropriate, including sample-size justification, bias-control strategy, missing-data handling, and statistical method specification. A high POMP score, however, does not indicate that the research question is clinically important, biologically plausible, or free from unmeasured confounding. These dimensions necessarily depend on domain expertise, causal reasoning, data quality, and external evidence, and lie beyond the scope of checklist-based evaluation. Accordingly, POMP is best understood as a process-level quality indicator, not a comprehensive measure of scientific merit.

Second, regarding the predictability evaluation of the execution analysis agent, we utilized the 22-item SAP evaluation framework (see Supplementary File 1, Material S10)—operationalized from the statistical analysis section (see Supplementary File 1, Material S4, Chapter 7) of the planning design agent's protocol—as the comparative scoring rubric. We uploaded this rubric alongside the code and statistical results output by the execution analysis agent to the LLM (e.g., Gemini 2.5 Pro). Based on the established prompt engineering (see Supplementary File 1, Material S11), the system performed a quantitative performance evaluation of the agent's code and statistical outputs.

Crucially, to assess the objectivity and validity of the evaluation results produced by the LLM (e.g., Gemini 2.5 Pro), we selected 10 senior experts in epidemiology or biostatistics independent of the study team. We paired these experts 1:1 based on professional background (epidemiology or biostatistics) and similarity in age (within 1−5 years), resulting in two groups of five experts each. For each pair, we generated two random numbers using a random number table and ranked them; the expert with Rank 1 was assigned to the experimental group, and the expert with Rank 2 to the control group.

Regarding the expert-validation sample size, we performed an exact noncentral-t power calculation for a paired t-test with two-sided α = 0.05, 1 − β = 0.95, and a conservative anticipated paired standardized effect size of Cohen's dz = 1.50, chosen below the effect observed in the 300-pair benchmark comparison (overall dz = 2.50; stratum-specific dz = 2.30, 3.20, and 3.59). The minimum required sample was eight article–protocol pairs; we selected nine pairs to support balanced stratified random sampling, with three articles independently and randomly selected from each JCR quartile stratum (Q1, Q2, and Q3). With n = 9 and dz = 1.50, the resultant power was approximately 0.975.

1. The five experts in the experimental group evaluated the performance of the intelligent design protocols (generated for the original published articles), using the original published articles as background reference and strictly adhering to the aforementioned 32-item STROBE_CSS criteria.

2. Conversely, the control group experts directly evaluated the research design and analytical performance of the original published articles, using the same criteria and the original published articles as background reference.

For the comparison of evaluation scores between paired experts, we employed paired t-tests. For reliability analysis, we used ICC(A,5) (two-way random, average measures) to assess Inter-Rater Reliability (IRR) among the five experts within the experimental (or control) group. To assess automated–expert calibration, we compared each object's automated POMP score (Gemini 2.5 Pro) with the corresponding five-rater expert-consensus POMP score across 18 objects. Agreement was summarized using Pearson r, Lin's CCC, mean bias, and MAE. For the IRR analysis, ICC (A,5) was used, and relaxed item-level agreement was additionally reported because protocol scores approached the scale ceiling.

Computability evaluation
To assess the operational feasibility of AgenMed_CSS under standard computational resource constraints (MacBook Air, 13-inch, 2025; Apple M4; 16 GB RAM; 512 GB SSD), the system was deployed on a standard laptop configuration. The system was run using real-time API calls to LLMs such as Claude-Sonnet-4 and ChatGPT-5. Operational logs were monitored to track metrics including the first-pass correctness rate of code generation for major analysis stages, the efficiency of automatic repair, the accuracy of key method code calls, the overall process success rate, and total execution time. These metrics were used to evaluate the transparency, auditability, and engineering robustness of the computational process, verifying its practical utility (see Supplementary File 1, Material S13).

Output consistency evaluation
This analysis was inspired by the PCS perturbation principle but was not a full PCS stability analysis: the underlying dataset, variable definitions, model specifications, and analytical decisions were not perturbed. Inferential tests were used only as exploratory comparisons and were not interpreted as evidence of stability or equivalence.

1. Planning output consistency: For a single standardized observational research project (see Supplementary File 1, Material S14), the planning design agent iteratively called four widely used LLMs (Claude-Sonnet-4, Gemini 2.5 Pro, ChatGPT-5, and DeepSeek-V3), 50 times each. Welch's ANOVA was conducted to compare POMP scores across backends, with Games–Howell post-hoc tests for pairwise comparisons. Results are reported descriptively as group means, 95% CIs, and absolute between-backend differences.

2. Execution output consistency: Targeting the statistical analysis section of the generated protocol (see Supplementary File 1, Material S4, Chapter 7), the execution analysis agent iteratively called two LLMs (Claude-Sonnet-4 and ChatGPT-5) 50 times each. Welch's t-test was used to compare the performance of the output statistical analysis code and results based on the 22 SAP evaluation metrics.

Validation research design
The validation study links each comparison object with a corresponding evaluation metric and a pre-specified statistical method. The published-article comparison assesses whether agent-generated protocols achieve higher STROBE_CSS completeness than matched peer-reviewed cross-sectional studies, evaluated using POMP scores and paired t-tests. The cross-LLM comparison assesses whether outputs remain consistent across different LLM backends, evaluated using POMP scores, execution-fidelity scores, and Welch's ANOVA. The unconstrained human-AI collaboration comparison assesses whether the structured agentic workflow outperforms general-purpose LLM use without the Agentic T-STROBE framework, evaluated using POMP scores and independent-sample t-tests. These components are described below.

Comparison objects

Published articles
A systematic search was conducted in the PubMed database for cross-sectional study literature published from 2020 to 2025 in JCR Q1, Q2, and Q3 journals (see Supplementary File 1, Material S15). Using simple random sampling, 100 articles were selected from each partition (total n = 300). Based on the background, objectives, and materials and methods of each article, the planning design agent generated a complete intelligent design protocol under the constraints of the Agentic_STROBE_CSS prompt engineering framework developed in this study (see Supplementary File 1, Materials S1 and S2).

Cross-LLM output consistency
On the one hand, for the same observational study project (see Supplementary File 1, Material S14), the planning design agent generated 50 intelligent design protocols each (total n = 200) by iterating through calls to four LLMs: Claude-Sonnet-4, Gemini 2.5 Pro, ChatGPT-5, and DeepSeek-V3, under the Agentic_STROBE_CSS prompt constraints (see Supplementary File 1, Material S1).

On the other hand, for the statistical analysis section of the generated protocols (see Supplementary File 1, Material S4), the execution analysis agent generated statistical analysis code and results 50 times each (total n = 100) by iterating through calls to Claude-Sonnet-4 and ChatGPT-5.

Unconstrained human-AI collaboration
Thirty-two students majoring in medical dataology (19 seniors and 13 graduate students), trained in epidemiology, biostatistics, and data science, were recruited. They were tasked with independently generating a design protocol for the same observational study project (see Supplementary File 1, Material S14) using any general-purpose LLM (platform and model unconstrained). No Agentic_STROBE_CSS prompt engineering framework (see Supplementary File 1, Material S1) or any templates, scoring criteria, or example schemes were provided. The resulting schemes served as the baseline comparator representing the currently prevalent unconstrained human-AI collaboration mode.

Evaluation metrics

STROBE_CSS methodological completeness
Each original published article (or intelligent design protocol) was evaluated using the 32-item STROBE_CSS scoring rubric (see Supplementary File 1, Material S8). This rubric assesses reporting completeness and the methodological appropriateness of predefined design and analysis elements, rather than the intrinsic scientific merit of the research question. Each item was scored on a three-point scale: 0 = missing or inappropriate; 1 = partially achieved; 2 = fully achieved and methodologically correct. The raw score $ {{R}}_{{i}} $ for the $ i $^th article (or protocol) is defined as: $ {{R}}_{{i}}=\displaystyle\sum\limits_{{j=1}}^{32}{{s}}_{{ij}} $, where $ s_{ij}\in\left\{0,1,2\right\} $, with a maximum possible score of 64. To facilitate comparison, raw scores were linearly standardized to a 0–100 scale using the POMP (percent of maximum possible) method^[36]:

$ POMP_i=100\times\dfrac{R_i}{64} $ (1)

Higher values indicate superior performance by the planning design agent. Scoring was executed by an LLM (e.g., Gemini 2.5 Pro) based on established prompt engineering (see Supplementary File 1, Material S9).

Statistical execution compliance

The execution fidelity was quantitatively evaluated using 'achieved/not achieved' ratings against the 22 evaluation metrics of the SAP (see Supplementary File 1, Material S10) derived from the statistical analysis section of the intelligent design protocol. The compliance rate (on a 100-point scale) for a single run $ {r} $ is defined as:

$ {{A}}_{{r}}=100\times \dfrac{1}{22}\sum\limits_{{k=1}}^{22}{{c}}_{{rk}} $

(2)

where $ c_{rk}\in\left\{0,1\right\} $ represents the judgment result for the $ {k} $^th checkpoint in the $ {r} $^th run. Higher values indicate better performance by the execution analysis agent. Scoring was executed by an LLM (e.g., Gemini 2.5 Pro) based on established prompt engineering (see Supplementary File 1, Material S11).

Statistical analysis

All statistical comparisons were two-sided with a significance threshold of α = 0.05.

Paired comparisons (intelligent design protocol vs. published articles)
For the published-article comparison, paired t-tests were used to compare the intelligent design protocols with the original published articles. Effect sizes (Cohen's dz) and 95% confidence intervals (CI) for the mean difference were calculated. Forest plots were generated to present item-level mean scores and their 95% CIs.

Cross-model consistency
For cross-LLM output consistency, due to equal sample sizes but potential heterogeneity of variance, Welch's ANOVA was used for global testing, with Games–Howell tests for post-hoc pairwise comparisons. Results reported include p-values, maximum absolute inter-group differences, group means, and 95% CIs.

Execution performance
For the execution analysis agent, Welch's t-test was used to compare performance on the 22 SAP metrics between the two LLMs (Claude-Sonnet-4 vs ChatGPT-5). Item-level achievement proportions and 95% CIs were reported.

System vs human-AI
Independent sample t-tests were used to compare differences between the four LLM-generated protocol groups and the unconstrained human-AI collaboration group, reporting POMP scores and 95% CIs for both groups.

Computability
Descriptive statistics were reported for the AgenMed_CSS computability metrics, including first-pass code generation correctness, automatic repair efficiency, key method code call accuracy, process success rate, and execution time (see Supplementary File 1, Material S13).

Discussion

The empirical results presented in this study not only validate the efficacy of a novel AI system but also pose profound questions regarding current research paradigms, the foundations of scientific trust, and the future evolution of inquiry.

From descriptive guidelines to executable contracts
Our findings support the value of front-shifting STROBE guidelines from a passive reporting checklist to a set of active design constraints. When these constraints are embedded within an agentic execution framework, methodological norms are transformed from suggestions into executable contracts. The system does not merely recommend best practices; it procedurally enforces them.

The comparison with 300 published articles (Fig. 3) reveals that the 'artisanal mode' of science, which relies on human experts to voluntarily adhere to guidelines, suffers from systemic, substantial, and persistent execution biases. AgenMed_CSS fundamentally resolves this by externalizing and proceduralizing tacit expert knowledge. The agent does not simulate a perfect scientist but rather faithfully executes a methodological checklist that is more comprehensive and tireless than human cognitive capacity allows, thereby raising the baseline of checklist-concordant methodological execution.

This baseline, however, refers specifically to the process-level quality of methodological execution. It does not imply that the system can judge whether a research question is clinically important, biologically plausible, or theoretically significant. That judgment remains part of the conceptual layer of scientific inquiry and requires human expertise.

Peer review as a narrative filter: implications for process auditing
The systematic superiority of AgenMed_CSS over published articles provides a unique, large-scale empirical perspective for reconsidering the essential function of peer review. Our results indicate (Fig. 3) that successful publication in respected journals is not synonymous with methodological completeness. This observation does not seek to fault individual reviewers or editors but rather illuminates an intrinsic structural blind spot within the peer review system.

Current peer review functions primarily as a narrative filter rather than a process auditor^[37−39]. It is adept at evaluating whether a manuscript's narrative is coherent, its question significant, and its conclusions novel. However, it possesses neither the capacity nor the design to audit the generative process of the research itself, verifying whether every decision from initial conception to final data analysis adhered to best practices. Reviewers evaluate static text, not the dynamic execution path replete with decision branches. By generating a complete computational lineage containing all intermediate artifacts and decision logs (see Supplementary File 1, Material S7), AgenMed_CSS addresses this gap in process auditing, offering a novel dimension for assessing research credibility.

Process as proof: a new foundation for trust
Under the artisanal paradigm, trust in scientific research relies heavily on proxy metrics: author reputation, institutional ranking, and journal impact factor. This model of trust is indirect and opaque. The emergence of AgenMed_CSS signals the possibility of a more robust foundation for trust: Process as Proof^[37,40,41].

When the entire trajectory of a study—from every design decision in the protocol to every line of analysis code and its execution result—is recorded in a complete, immutable format (see Supplementary File 1, Material S7), the locus of trust shifts from faith in the person to verification of the process. The academic community no longer simply believes that an author did the right thing; they can directly review the computational records and reproduce the analysis workflow with a single click. This trust, grounded in a verifiable process, is more objective, transparent, and robust than any reputation-based trust.

Cognitive stratification: the scientist as architect of inquiry
Concerns regarding AI displacing scientists are widespread; however, our research points toward a fundamentally different future. The comparison with the unconstrained human-AI collaboration mode (Fig. 5) reveals that simply equipping humans with powerful tools does not resolve the underlying issues. True paradigmatic progress lies in a re-delineation of the boundaries of responsibility between humans and machines, a concept we term Cognitive Stratification.

In this new model, the cognitive tasks of machines and humans are clearly stratified and decoupled. The execution layer belongs to the agentic system and encompasses methodological steps that are procedural, error-intolerant, and require high fidelity and consistency; in this layer, the machine serves as the guarantor of the methodological foundation. The conceptual layer remains the domain of the human scientist and includes formulating insightful scientific questions, constructing theoretical frameworks, interpreting complex background knowledge, critically evaluating research results, and making final value judgments.

This stratification represents supervision rather than replacement. In AgenMed_CSS, it is operationalized through three researcher-facing stages: structured question framing at the outset, review and refinement of the generated protocol and SAP before execution, and post-execution inspection of code, tables, figures, and audit logs. This stratification also defines the boundary of the present evaluation. The agentic system can enforce whether a protocol specifies the required design and analysis components, but it cannot determine whether the underlying research question is worth pursuing. The formulation of meaningful hypotheses, assessment of biological plausibility, appraisal of clinical relevance, and final interpretation of results remain responsibilities of human investigators.

Limitations and generalizability
This study has limitations. First, the current system targets cross-sectional designs within observational research; its applicability to cohort studies, case-control studies, and randomized controlled trials remains to be developed and verified. Second, the STROBE_CSS-based POMP score measures checklist-based methodological completeness, not the intrinsic merit of the research question. A protocol may be complete and methodologically well-specified while still addressing a clinically unimportant or biologically implausible question. This reflects the intended boundary of Agentic T-STROBE, which targets the execution layer of research rather than the conceptual layer of scientific judgment. Finally, the cross-backend output consistency analyses did not constitute full PCS stability testing, as the underlying dataset, variable definitions, model specifications, and analytical decisions were not perturbed to test the robustness of substantive scientific conclusions.

Nevertheless, the core paradigm of this study—forward-shifted guidelines, agentic execution, and PCS evaluation—is highly modular and generalizable. The underlying architecture is domain-agnostic. By replacing the underlying STROBE rule base with CONSORT guidelines for randomized trials^[42] or ARRIVE guidelines for animal research^[43], this paradigm could be seamlessly migrated to other research fields. This universality suggests its potential to serve as a general-purpose infrastructure for systematic scientific inquiry across disciplines.

Supplementary File 1 Supplementary materials to this study.
Supplementary File 2 Source data for Figs 3−6 and Table 1 in this study.
Video S1 n8n workflow and modular prompt chains of the AgenMed_CSS planning design agent.
Video S2 End-to-end demonstration of AgenMed_CSS: research protocol generation and statistical analysis.

[1]	Errington TM, Mathur M, Soderberg CK, Denis A, Perfito N, et al. 2021. Investigating the replicability of preclinical cancer biology. eLife 10:e71601 doi: 10.7554/eLife.71601 CrossRef Google Scholar
[2]	Camerer CF, Dreber A, Holzmeister F, Ho TH, Huber J, et al. 2018. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour 2:637−644 doi: 10.1038/s41562-018-0399-z CrossRef Google Scholar
[3]	Holzmeister F, Johannesson M, Camerer CF, Chen Y, Ho TH, et al. 2025. Examining the replicability of online experiments selected by a decision market. Nature Human Behaviour 9:316−330 doi: 10.1038/s41562-024-02062-9 CrossRef Google Scholar
[4]	Chakravorti T, Koneru S, Rajtmajer S. 2025. Reproducibility and replicability in research: What 452 professors think in Universities across the USA and India. PLoS One 20:e0319334 doi: 10.1371/journal.pone.0319334 CrossRef Google Scholar
[5]	Angelopoulos A, Cahoon JF, Alterovitz R. 2024. Transforming science labs into automated factories of discovery. Science Robotics 9:eadm6991 doi: 10.1126/scirobotics.adm6991 CrossRef Google Scholar
[6]	Niederer SA, Sacks MS, Girolami M, Willcox K. 2021. Scaling digital twins from the artisanal to the industrial. Nature Computational Science 1:313−320 doi: 10.1038/s43588-021-00072-5 CrossRef Google Scholar
[7]	Seifrid M, Pollice R, Aguilar-Granda A, Chan ZM, Hotta K, et al. 2022. Autonomous chemical experiments: challenges and perspectives on establishing a self-driving lab. Accounts of Chemical Research 55:2454−2466 doi: 10.1021/acs.accounts.2c00220 CrossRef Google Scholar
[8]	von Kortzfleisch VT, Ambrée O, Karp NA, Meyer N, Novak J, et al. 2022. Do multiple experimenters improve the reproducibility of animal studies? PLoS Biology 20:e3001564 doi: 10.1371/journal.pbio.3001564 CrossRef Google Scholar
[9]	von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, et al. 2007. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335:806−808 doi: 10.1136/bmj.39335.541782.AD CrossRef Google Scholar
[10]	Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, et al. 2007. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Epidemiology 18:805−835 doi: 10.1097/EDE.0b013e3181577511 CrossRef Google Scholar
[11]	Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, et al. 2015. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Medicine 12:e1001885 doi: 10.1371/journal.pmed.1001885 CrossRef Google Scholar
[12]	Vandenbroucke JP. 2007. The making of STROBE. Epidemiology 18:797−799 doi: 10.1097/EDE.0b013e318157725d CrossRef Google Scholar
[13]	Pouwels KB, Widyakusuma NN, Groenwold RHH, Hak E. 2016. Quality of reporting of confounding remained suboptimal after the STROBE guideline. Journal of Clinical Epidemiology 69:217−224 doi: 10.1016/j.jclinepi.2015.08.009 CrossRef Google Scholar
[14]	Lu MY, Chen B, Williamson DFK, Chen RJ, Zhao M, et al. 2024. A multimodal generative AI copilot for human pathology. Nature 634:466−473 doi: 10.1038/s41586-024-07618-3 CrossRef Google Scholar
[15]	Zhang J, Liu L, Xiang P, Fang Q, Nie X, et al. 2024. AI co-pilot bronchoscope robot. Nature Communications 15:241 doi: 10.1038/s41467-023-44385-7 CrossRef Google Scholar
[16]	Zhang Z, Ren Z, Hsu CW, Chen W, Hong ZW, et al. 2025. A multimodal robotic platform for multi-element electrocatalyst discovery. Nature 647:390−396 doi: 10.1038/s41586-025-09640-5 CrossRef Google Scholar
[17]	Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, et al. 2023. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. npj Digital Medicine 6:75 doi: 10.1038/s41746-023-00819-6 CrossRef Google Scholar
[18]	Piccolo SR, Denny P, Luxton-Reilly A, Payne SH, Ridge PG. 2023. Evaluating a large language model's ability to solve programming exercises from an introductory bioinformatics course. PLoS Computational Biology 19:e1011511 doi: 10.1371/journal.pcbi.1011511 CrossRef Google Scholar
[19]	Dennstädt F, Zink J, Putora PM, Hastings J, Cihoric N. 2024. Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain. Systematic Reviews 13:158 doi: 10.1186/s13643-024-02575-4 CrossRef Google Scholar
[20]	Tang L, Sun Z, Idnay B, Nestor JG, Soroush A, et al. 2023. Evaluating large language models on medical evidence summarization. npj Digital Medicine 6:158 doi: 10.1038/s41746-023-00896-7 CrossRef Google Scholar
[21]	Alber DA, Yang Z, Alyakin A, Yang E, Rai S, et al. 2025. Medical large language models are vulnerable to data-poisoning attacks. Nature Medicine 31:618−626 doi: 10.1038/s41591-024-03445-1 CrossRef Google Scholar
[22]	Vaccaro M, Almaatouq A, Malone T. 2024. When combinations of humans and AI are useful: a systematic review and meta-analysis. Nature Human Behaviour 8:2293−2303 doi: 10.1038/s41562-024-02024-1 CrossRef Google Scholar
[23]	Hua HU, Kaakour AH, Rachitskaya A, Srivastava S, Sharma S, et al. 2023. Evaluation and comparison of ophthalmic scientific abstracts and references by current artificial intelligence chatbots. JAMA Ophthalmology 141:819−824 doi: 10.1001/jamaophthalmol.2023.3119 CrossRef Google Scholar
[24]	Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, et al. 2023. Large language models in medicine. Nature Medicine 29:1930−1940 doi: 10.1038/s41591-023-02448-8 CrossRef Google Scholar
[25]	Roustan D, Bastardot F. 2025. The clinicians' guide to large language models: a general perspective with a focus on hallucinations. Interactive Journal of Medical Research 14:e59823 doi: 10.2196/59823 CrossRef Google Scholar
[26]	Dip SA, Mallick D, Shuvo UA, Soumma SB, Rafsani F, et al. 2026. Large language model agents for biological intelligence across genomics, proteomics, spatial biology, and biomedicine. Briefings in Bioinformatics 27(2):bbag110 doi: 10.1093/bib/bbag110 CrossRef Google Scholar
[27]	Bu D, Sun J, Li K, He Z, Huang W, et al. 2026. Empowering AI data scientists using a multi-agent LLM framework with self-evolving capabilities for autonomous, tool-aware biomedical data analyses. Nature Biomedical Engineering Ahead of print doi: 10.1038/s41551-026-01634-6 CrossRef Google Scholar
[28]	Tzanis E, Adams LC, Akinci D'Antonoli T, Bressem KK, Cuocolo R, et al. 2026. Agentic systems in radiology: principles, opportunities, privacy risks, regulation, and sustainability concerns. Diagnostic and Interventional Imaging 107(1):7−16 doi: 10.1016/j.diii.2025.10.002 CrossRef Google Scholar
[29]	Xu Q, Soto C, Shahnawaz M, Liu X, Jiang X, et al. 2025. Multi agent large language models for biomedical hypothesis generation in drug combination discovery. iScience 28(12):113984 doi: 10.1016/j.isci.2025.113984 CrossRef Google Scholar
[30]	Yu B, Kumbier K. 2020. Veridical data science. Proceedings of the National Academy of Sciences USA 117:3920−3929 doi: 10.1073/pnas.1901326117 CrossRef Google Scholar
[31]	Rosner B. 2015. Fundamentals of biostatistics. 8th ed. Boston: Cengage Learning. 888 pp
[32]	Wang L, Chen X, Deng XW, Wen H, You MK, et al. 2024. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. npj Digital Medicine 7:41 doi: 10.1038/s41746-024-01029-4 CrossRef Google Scholar
[33]	Chen S, Gao M, Sasse K, Hartvigsen T, Anthony B, et al. 2025. When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior. npj Digital Medicine 8:605 doi: 10.1038/s41746-025-02008-z CrossRef Google Scholar
[34]	Adams AD, Benner RS, Riggs TW, Chescheir NC. 2018. Use of the STROBE checklist to evaluate the reporting quality of observational research in obstetrics. Obstetrics and Gynecology 132:507−512 doi: 10.1097/AOG.0000000000002689 CrossRef Google Scholar
[35]	Ziemann S, Paetzolt I, Grüßer L, Coburn M, Rossaint R, et al. 2022. Poor reporting quality of observational clinical studies comparing treatments of COVID-19: a retrospective cross-sectional study. BMC Medical Research Methodology 22:23 doi: 10.1186/s12874-021-01501-9 CrossRef Google Scholar
[36]	Cohen P, Cohen J, Aiken LS, West SG. 1999. The problem of units and the circumstance for POMP. Multivariate Behavioral Research 34:315−346 doi: 10.1207/S15327906MBR3403_2 CrossRef Google Scholar
[37]	Chambers CD, Tzavella L. 2022. The past, present and future of registered reports. Nature Human Behaviour 6:29−42 doi: 10.1038/s41562-021-01193-7 CrossRef Google Scholar
[38]	Proctor DM, Dada N, Serquiña A, Willett JLE. 2023. Problems with peer review shine a light on gaps in scientific training. mBio 14:e0318322 doi: 10.1128/mbio.03183-22 CrossRef Google Scholar
[39]	Hamilton DG, Hong K, Fraser H, Rowhani-Farid A, Fidler F, et al. 2023. Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data. BMJ 382:e075767 doi: 10.1136/bmj-2023-075767 CrossRef Google Scholar
[40]	Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, et al. 2022. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine 28:924−933 doi: 10.1038/s41591-022-01772-9 CrossRef Google Scholar
[41]	Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, et al. 2024. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 385:e078378 doi: 10.1136/bmj-2023-078378 CrossRef Google Scholar
[42]	Schulz KF, Altman DG, Moher D. 2010. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 340:c332 doi: 10.1136/bmj.c332 CrossRef Google Scholar
[43]	Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. 2010. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biology 8:e1000412 doi: 10.1371/journal.pbio.1000412 CrossRef Google Scholar

Section	Metric	All (n = 18)	Articles (n = 9)	Protocols (n = 9)
A. Expert consensus POMP scores	Mean ± SD (%)	−	75.03 ± 12.11	95.10 ± 2.39
	95% CI	−	65.72–84.35	93.27–96.94
B. Inter-rater reliability	ICC (A,5) (95% CI)	−	0.92 (0.79, 0.98)	−0.25 (−0.99, 0.55)
−	Relaxed agreement	−	76.70%	91.00%
C. Automated–expert calibration	Pearson r	0.91 (p < 0.001)	−	−
−	Lin's CCC	0.83	−	−
−	Mean bias	−3.3	−9.58	2.99
−	MAE	6.74	10.49	2.99
Data are presented as mean ± SD or point estimates with 95% confidence intervals (CIs). Section A quantifies the methodological completeness gap, where five-rater expert consensus confirms that agent-generated protocols outperform matched peer-reviewed articles. Section B reports inter-rater reliability and relaxed item-level agreement. The anomalous ICC value for the protocol group reflects a ceiling effect caused by restricted between-object variance, as experts consistently rated these protocols near the maximum score; it should therefore be interpreted together with the relaxed agreement metric. Section C reports automated–expert calibration at the object level using Pearson r, Lin's concordance correlation coefficient (CCC), mean bias, and MAE. Abbreviations: CCC, Lin's concordance correlation coefficient; ICC, intraclass correlation coefficient; MAE, mean absolute error; POMP, percent of maximum possible score. The source data is provided in Supplementary File 2.

{{lists.name}}

Agentic T-STROBE: a front-shifted STROBE paradigm for observational research

Abstract

Supplementary information

Rights and permissions

References

About this article

Cite this article

Special Issue

Article Metrics

Access History

Other Articles By Authors