HHS AI Research Misconduct Guidance: Old Rules for New Evidence

A researcher uses generative artificial intelligence (AI) to process data, draft a grant application or assemble a literature review. If the work is later challenged, investigators may need to reconstruct what the tool produced, what the researcher checked and whether the underlying records still exist. That is the practical challenge in new guidance from the Department of Health and Human Services’ Office of Research Integrity (ORI) on AI use in public health service-funded research.

ORI applies the existing research misconduct framework to generative AI. A finding requires a significant departure from accepted practices in the relevant research community, conduct committed intentionally, knowingly or recklessly, and proof by a preponderance of the evidence. The guidance offers nonbinding recommendations under revised regulations that became applicable to new allegations in January. AI creates no separate category of misconduct, but it complicates how institutions establish accepted practice, assess a researcher’s state of mind and preserve proof.

Researchers should identify the AI tools used in research and in preparing manuscripts and grants, and explain how they used them, ORI said. Disclosure can help establish reproducibility and defend against an allegation. It does not excuse conduct that significantly departs from accepted practice. Crowell & Moring, in an analysis of the guidance, advised institutions to document the inputs, outputs and verification steps as well as the tool’s name.

Determining accepted practice is itself a challenge. Research fields differ in their approaches to AI, while policies issued by funding agencies can also inform the standard. Crowell pointed to National Institutes of Health restrictions on grant applications “substantially developed” by AI and on peer reviewers using generative AI to analyze applications or prepare critiques. Institutions must determine whether the particular use at issue was a significant departure from the practices governing that work.

An allegation of misconduct will proceed through the ordinary institutional process, per the ORI. A credible, sufficiently specific allegation within the regulation’s scope may warrant an inquiry—an initial review to decide whether a full investigation is justified. The institution must notify the respondent and obtain necessary records and evidence. If preliminary fact-finding indicates the allegation has substance, the institution begins an investigation and notifies ORI.

At the investigation stage, committees must include appropriate scientific expertise in the respondent’s field. ORI suggested adding AI specialists who can explain a tool’s functions and limits. Investigators should consider all available evidence, including research records, source materials, intermediate files and other indicators of whether experiments occurred as described. Automated plagiarism or manipulation detectors may provide a lead, but ORI advises against using their output as the sole evidence; plagiarism detectors may miss copied material in AI-generated scientific text.

Some AI errors defy simple classification. A fabricated citation in a grant application’s background section may fall outside the regulation’s definition of fabricated data or results because it is not part of the scientific inquiry. In a literature review, on the other hand, where references are part of the scientific inquiry, false citations may qualify. AI-generated text or ideas can amount to plagiarism if they reproduce another person’s work without credit. Researchers remain responsible for checking both citations and the provenance of material they use.

Evidence preservation presents another challenge. Prompts, outputs and processing records may be scattered across personal devices and external platforms, with some session histories deleted automatically. Institutions generally must retain the institutional record and sequestered evidence for seven years after a misconduct proceeding ends. Intentionally or knowingly destroying relevant records after learning of an allegation can itself serve as evidence of misconduct.

Crowell recommended updating disclosure and training policies, checking AI vendors’ retention practices, preserving prompts and model information, and identifying AI experts before cases arise. It also urged research integrity officers to route concerns carefully. An AI-related privacy or confidentiality breach does not necessarily meet the definition of research misconduct, for instance.

Separately, NIH may pursue grant-management remedies, including suspension or termination, while an ORI misconduct matter proceeds. For institutions, per Crowell, the immediate task is to make AI use verifiable before an allegation makes its history difficult to recover.