Paperpile Citation Checker in 2026: What It Catches and What Slips Through

9 min readUpdated Nov 17, 2026

Paperpile's free Citation Checker flags wrong references in a BibTeX file before you submit a preprint. Here is what that covers, and where a second check helps.

Citely
Paperpile Citation Checker in 2026: What It Catches and What Slips Through

Paperpile has announced a free Citation Checker that takes a BibTeX file, verifies each reference in real time, and lets you download a corrected file or share a results URL, according to Paperpile's own announcement. The problem it targets is documented. A July 2026 arXiv audit of ICLR, ICML, NeurIPS and USENIX Security papers found that roughly one in twenty NeurIPS and USENIX Security papers from 2025 contained at least two likely hallucinated citations, even though the reference-level rate stayed under 1%. If you are about to upload a preprint to arXiv or send a thesis chapter to your supervisor, a free pre-flight check on your .bib file is worth five minutes. This guide covers what the tool is built for, which kinds of reference errors a BibTeX-level check is well placed to catch, which ones need a different kind of check, and when Paperpile alone is enough.

Why a BibTeX check exists at all

The failure everyone worries about is the invented reference: a plausible author list, a plausible title, a DOI that leads nowhere. It is not hypothetical. In a 2023 study in Scientific Reports, William Walters and Esther Wilder asked ChatGPT to write 84 literature reviews across 42 topics and checked all 636 bibliographic citations. They found that 55% of the GPT-3.5 citations and 18% of the GPT-4 citations were fabricated, and that 43% of the real GPT-3.5 citations and 24% of the real GPT-4 citations contained substantive errors (Walters & Wilder, 2023). Those numbers describe two specific 2023 models on a specific task. They are not a current hallucination rate for any tool you use today.

The more recent evidence is about what survives into the literature. Russinovich, Siva Kumar and Salem built an open-source verification pipeline called RefChecker and ran it over papers from top ML and security venues. Their finding, in Phantom References (arXiv:2607.00738, July 2026), is that hallucinated citations do pass peer review, including in award-winning papers, and that auditing a whole venue is cheap, on the order of cents per paper.

Two things follow for a researcher. First, reviewers are not your safety net. Second, the people most exposed are those whose bibliography is generated, merged or copy-pasted from several sources at the last minute, which is exactly when a .bib file gets assembled.

What Paperpile's Citation Checker is, as publicly described

We could not load Paperpile's tool page or its launch blog post when we researched this article in October 2026 (the pages returned redirect errors), so everything below comes from Paperpile's public posts on X and its main site, not from hands-on testing of every edge case. Check the Citation Checker page for the current details before you rely on a specific behavior.

What Paperpile has said publicly: the tool is free, you upload a BibTeX file, it verifies every reference in real time, and you can download the updated file or share the URL (Paperpile on X). A companion post says it was built because hallucinated citations are showing up in preprints (Paperpile on X).

What we could not confirm: which databases it queries, how it labels a reference it cannot match, and how it treats books, theses or non-English sources. That is not a criticism. It is the list of questions to answer yourself by running a handful of your own entries through it, including one you know is wrong.

The reference manager is a different product

Paperpile is, first, a reference manager. Its feature page describes finding incomplete references and automatically looking up missing data online, resolving duplicates, and keeping journal names and abbreviations consistent. Those are hygiene features. They make a library tidy and consistent. They are not the same job as asking whether a given entry corresponds to a real paper, which is what a verification step does.

Paperpile's pricing page, checked 2026-11-02, lists a 30-day free trial and two individual plans billed annually: Regular at $8.30 per month and Expert at $11.50 per month, shown as $4.15 and $5.75 with the page's 50% academic discount. Prices may change; we re-check monthly. The Citation Checker is described as a free standalone tool, so you do not need a subscription to run a .bib file through it.

What a BibTeX-level check is well placed to catch

A .bib file is structured. Each entry has a type, a key, a title, authors, a year and often a DOI. That structure suits automated lookup, and it makes some errors easy to find.

Entries that match no record

If a title, author list and year match nothing in the bibliographic databases the checker uses, that entry is a candidate for a fabricated reference. This is the case most people fear, and it is the one a structured lookup handles best.

Broken or wrong DOIs

A malformed DOI or one that resolves to a different paper than the entry describes is a strong signal. Note that a DOI resolving at all proves little. It only proves the DOI exists, not that it belongs to the paper in your entry.

Missing fields

Entries with no year, no venue or truncated authors are common in exported libraries. A check that fills or flags them improves the bibliography even when nothing is fabricated.

Duplicates and inconsistent keys

When you merge .bib files from co-authors, the same paper often appears twice under different keys. That is a formatting problem, but it is the kind that makes a reference list look careless.

What slips through any single check

This section is the one to read before you tell a co-author the bibliography is clean. These failure modes are hard for any one tool, Paperpile's or ours, to settle completely.

The reference is real but the details are wrong

A real paper cited with the wrong year or a missing co-author is the quietest failure. The title matches, the DOI resolves, and a quick scan passes it. A reviewer who looks the paper up sees the mismatch immediately. Walters and Wilder's finding that a large share of the real citations still held substantive errors is the reason this category matters.

The paper exists but does not say that

Existence is not support. A verified reference can still be attached to a claim the paper never makes. No metadata check can fix that. You have to read the paper, or at least the abstract and the relevant section.

Sources that are not in the database

Theses, reports, books, conference papers without DOIs and non-English literature are unevenly covered by any bibliographic index. The Crossref REST API, for example, indexes only DOIs registered with Crossref (Crossref documentation). A legitimate report with no DOI can therefore look "unfound" without being fake. Treat an unmatched entry as a prompt to look, not as a verdict.

Text outside the .bib file

If the reference list lives in a Word document, a Google Doc, or the bottom of a ChatGPT answer, there is no .bib file to upload. You are either converting it first or checking it as pasted text.

An honest comparison

Paperpile Citation CheckerPaperpile reference managerCitely Citation Checker
Main jobVerify references in a BibTeX fileOrganize, cite and clean a libraryVerify a pasted reference list
InputBibTeX uploadImported or saved papersPasted reference text (up to 2,000 characters on Trial, 4,000 on paid plans)
OutputUpdated BibTeX file or shareable URL, per PaperpileFormatted citations and a tidy libraryVerified / Mismatch / Not Found per reference
Price (checked 2026-11-02)Free, per PaperpileRegular $8.30/month billed annually ($4.15 with the academic discount)3 free credits for new accounts
Best forLaTeX and BibTeX users before a preprintDay-to-day library managementLists that are not in BibTeX, or when you need to know why an entry failed

Paperpile prices are from paperpile.com/pricing, checked 2026-11-02. Prices may change; we re-check monthly. We have not independently benchmarked either checker's detection rate, and we do not publish an accuracy percentage for Citely.

How Citely fits

Citely starts from the same worry, a reference list you cannot fully trust, and checks it from a different direction. You paste the references into the Citation Checker, and each one is looked up against Crossref, OpenAlex, Semantic Scholar, Google Scholar, PubMed and arXiv, plus Wanfang and CNKI for Chinese papers, according to Citely.

The distinction that matters most for the failures above is the status labels. If a published record matches the reference, it is Verified. If no database holds a matching record, it is marked Not Found, and Citely shows the closest records so you can decide. If the paper exists but a detail such as the year, the authors or the DOI is wrong, it is marked Mismatch, with the wrong field named. That middle label is the one built for the "real paper, wrong details" case, because it tells you the reference exists and which part to fix instead of lumping it in with the fabricated ones. For lists that came out of an AI assistant, the AI Citation Checker page covers the typical patterns, and the Reference Checker page is aimed at a full bibliography.

Every new account gets 3 free credits, with no credit card required. Credits are what Citely uses for checks, and the credits a check will use are shown before it starts. See the pricing page for current paid plans.

If your entries are in BibTeX and you want to carry results into a manager, you can export to BibTeX and import the file into Zotero, Mendeley or EndNote. Citely does not replace a reference manager.

When you don't need a second tool

Manual checking, or Paperpile alone, is enough more often than a vendor blog usually admits.

  • You wrote the bibliography by hand from papers you have open. If every entry came from a PDF on your desktop, the risk is typos, not invention.
  • The list is short. Ten references can be checked by searching each title in Google Scholar and comparing authors and year. That takes about as long as uploading a file.
  • You live in LaTeX and BibTeX. If you are preparing a preprint, Paperpile's free checker is a sensible first pass, and it costs nothing to run.
  • You only need tidy formatting. Duplicates, abbreviations and missing fields are reference-manager problems, and Paperpile's library features are aimed at them.

What you should not skip is the cross-check on anything an AI wrote. Search the title yourself, compare the authors and year, and confirm that the paper supports the sentence it is attached to.

Key Takeaways

  1. Paperpile's Citation Checker is a free BibTeX-upload tool, per Paperpile's public announcement. Test it on a few of your own entries, including a deliberately wrong one, before you trust it on a full file.
  2. Reference-manager hygiene (duplicates, missing fields, journal abbreviations) and reference verification are different jobs. A tidy library can still contain a wrong citation.
  3. The hard failures are real papers with wrong details and real papers that do not support your claim. Walters and Wilder (2023) found substantive errors in a large share of even the real ChatGPT citations they checked.
  4. An unmatched entry is not automatically fake. Databases such as Crossref only index DOIs registered with Crossref, so reports, theses and some non-English sources need a manual look.
  5. Use a BibTeX check before a preprint, and a pasted-text check with explicit Verified, Mismatch and Not Found labels when the list is not in BibTeX or you need to know which field is wrong.

👉 Start free — 3 free credits, no credit card

Try Citely

Check citations and verify references faster.

Use Citely to spot fake, incomplete, or unsupported citations before submission.

Related Articles

Continue exploring topics you care about.