Using AI to Summarize Research Papers Responsibly

AI summaries can save real time when you are facing a stack of papers. They can also quietly introduce errors into your notes, your literature review and eventually your own published work. The difference between those outcomes is not which tool you use; it is the habits you bring to it.

This guide sets out the practices the POMODIARY team follows and recommends: what can go wrong, how to verify, how to cite, how to prompt, and how to handle confidential material and academic-integrity rules.

What can go wrong

Large language models generate fluent text that is statistically likely given their input. Most of the time a summary of a paper you supply is broadly accurate. The failures are specific and worth knowing by name.

Fabricated or distorted details

The most common problem is not a summary that is entirely wrong but one that is mostly right with a wrong detail: a sample size off by a factor of ten, an effect described as significant when the paper reports it was not, a dataset name that is plausible but incorrect. Because the surrounding text is correct, these errors are easy to miss.

Invented citations

When asked for related work or sources, models can produce references that look real (authors, year, journal, title) but do not exist, or that exist but do not say what the model claims. Never add a reference to your bibliography that you have not opened yourself.

Overstated conclusions

Papers hedge carefully: "associated with", "in this sample", "suggests". Summaries tend to strip those hedges, turning a correlation into a cause or a preliminary result into a finding. The hedges are often the most scientifically important words in the paper.

Missing context

A summary may omit the limitations section, the fact that the result only held for one subgroup, or that the paper is a preprint not yet peer reviewed. Absence is hard to notice because you cannot see what is not there.

Partial reading

Some tools process long PDFs in chunks, and tables, figures, equations and appendices may be read poorly or not at all. If a key result lives in a table, the summary may be paraphrasing the text around it instead.

A verification workflow

Treat every AI summary as a set of claims to check, not a set of facts. A practical routine:

Read the abstract yourself first. It takes two minutes and gives you a baseline to compare the summary against.
Ask for locations, not just answers. Prompt for the section, page or table where each claim appears, then go there.
Check every number. Sample sizes, effect sizes, percentages, p-values and dates go into your notes only after you have seen them in the PDF.
Check the direction and strength of claims. Compare the summary's verbs with the paper's verbs. Did "may reduce" become "reduces"?
Read the limitations paragraph. If the summary did not mention any limitations, that alone tells you something was dropped.
Mark your notes. Distinguish "verified in PDF" from "from AI summary, unchecked". A simple tag in your notes saves you from trusting your own past shortcuts.

Claim type	Risk if unchecked	Where to verify
Numbers and statistics	High	Results section, tables
Causal language	High	Discussion, abstract wording
Method details	Medium	Methods section
Related work and citations	High	Reference list, then the cited paper
General topic and motivation	Low	Introduction

Citing correctly

The rule is simple: cite the paper, never the summary. If a claim in your writing depends on a paper, you should have read the relevant part of that paper.

Cite the version you read. Preprints and published versions can differ, sometimes substantially. Note which one you used.
Quote from the source text. If you quote, copy from the PDF, not from the AI's paraphrase, and include the page number where your style guide requires it.
Do not launder secondary claims. If paper A says paper B found something, and you want to use that finding, read paper B.
Follow your field's guidance on AI disclosure. Many journals, conferences and universities now have rules on whether and how AI assistance must be disclosed. Check them; they vary and change.

Prompts that work

Vague prompts produce vague, confident summaries. Specific prompts produce answers you can check. Some patterns we find useful:

"List the paper's main claims. For each, give the section or table where it is supported."
"What is the sample or dataset, how large is it, and how was it selected? Quote the sentence that states this."
"What limitations do the authors themselves state? Quote them."
"What is the primary comparison or baseline, and what is the reported difference?"
"Explain this paragraph in simpler terms without adding information that is not in it."
"Which claims in the abstract are not directly supported by a figure or table?"

Asking for quotations is especially effective, because a quotation can be checked with a quick search of the PDF. If the quoted sentence is not there, you have found a hallucination.

Prompts to avoid: "Is this paper good?" and "Summarize the literature on X." The first invites an opinion with no basis you can inspect; the second invites invented references.

Privacy and unpublished manuscripts

Not every document is yours to upload. Before you put a PDF into any AI tool, ask who owns it and what they agreed to.

Peer review: Manuscripts sent for review are confidential. Many publishers and funding bodies restrict or forbid uploading them to AI tools. Read the reviewer instructions before doing anything.
Collaborators' drafts: Get explicit agreement before uploading a co-author's unpublished work.
Sensitive data: Papers or appendices containing patient, participant or proprietary data may be covered by ethics approvals, contracts or law.
Know the tool's data policy: Check whether uploads are stored, for how long, and whether they are used to train models. If you cannot find out, treat the tool as public.

When in doubt, work with published versions only, or with your own notes rather than the manuscript itself.

Academic integrity

Using AI to understand a paper is broadly accepted. Using it to produce work you present as your own understanding may not be, depending on your institution. A few principles hold in most settings:

Your analysis must be yours. A literature review is valuable because it reflects your judgment about which work matters and why. An AI summary cannot supply that judgment.
Disclose when required. Follow course, institutional and publisher rules on AI use, and when rules are unclear, ask.
Do not paste AI summaries into submitted work. Beyond integrity concerns, they will contain the errors described above.
Keep a trail. Notes that distinguish your reading from AI output make it easy to show your process if asked.

Where AI summaries genuinely help

None of this means avoiding AI. Used well, it makes you a broader and faster reader:

Discovery: scanning many new papers to decide which deserve your time.
Orientation: getting the vocabulary of an unfamiliar subfield before reading its key papers.
Clarification: unpacking a dense paragraph or an unfamiliar method.
Language support: reading passages side by side in your first language.

That is the role we designed ChatPaper.AI to play. Its daily issue of five plain-language summaries is a discovery layer, and its workspace and select-to-ask features help with clarification on a specific passage. The final step, checking the claim in the paper itself, is always yours. If you want a structured way to do that checking, see our separate guide on how to read a research paper with the three-pass method.

A short checklist

Before anything from an AI summary goes into your writing:

I have seen this claim in the original paper.
I have checked every number against the PDF.
The strength of the claim matches the paper's wording.
I am citing the version I actually read.
I was allowed to upload this document.
I have followed the AI disclosure rules that apply to me.

Six checks, a few minutes per paper, and AI becomes a reading aid instead of a source of quiet errors.