How this works
What follows is the whole process, plus some honest answers to the questions editorial teams and boards have actually asked me. If your editorial team or board has a question that isn't here, send it and I'll add it.
Seven stages
-
1 · You send me a URL
I look at your archive and work out which platform you publish on. Four publishers that I've already worked with can be handled without any new code — works on the WAC Clearinghouse, OJS, Janeway, and bepress/Digital Commons already have tooling designed to complete this process. Anything else outside these forums will get a bespoke downloader written for it (usually a day or two, longer if the archive is doing something genuinely strange with its URLs), which takes more time but isn't hard.
-
2 · I download your back catalog, slowly
The scraper waits a second between requests, and five to ten seconds where a site looks like it is running on a volunteer's budget. I respect robots.txt without exception, and my contact address rides along in every request I make.
-
3 · I find out what I'm actually dealing with
Older volumes are often scans with no text layer underneath them. This is the stage where that surfaces and where OCR gets added if the archive needs it. Every journal presents its own particular kind of mess, and the mess is usually in the oldest volumes. This work is slow, so if you are in this boat, let's talk timelines a bit as I end up doing much of this work by hand using my CrossRef Auditor.
-
4 · I extract the reference lists
If your journal publishes HTML galleys, EXCELLENT, because the markup already knows where one citation ends and the next begins. If your journal publishes PDFs, they go through GROBID, an open-source document-structuring library (it is very good and it is not magic — two-column scans and footnote-style citation give it trouble). After that comes a set of repair passes for the things humanities citations actually do: em dashes standing in for a repeated author, entries split across a page break, a notes section glued onto the last works-cited entry, running heads captured as though they were references, etc.
-
5 · Tooling then looks up every reference in Crossref
Where a reference matches a work that already has a DOI, that DOI gets attached and a live citation link comes into existence. Three checks have to pass before I'll accept a match: at least half the title words overlap, at least one author surname matches, and the publication year falls within one. The checks are deliberately strict, because a wrong DOI is worse than no DOI; it asserts an identity between two different works and pollutes every analysis downstream of it.
-
6 · I audit the whole file by hand
Crossref's validator checks that your deposit is well-formed. It does not check whether the contents make sense, which means a paragraph of your own body text can pass validation cleanly and land in the permanent record as a citation. I wrote a separate tool for exactly this problem and released it under the GPL at github.com/justalewis/crossref-references-deposit-auditor, so you can read what it checks. It flags the citations that look wrong, repairs most of them automatically, and puts the remainder in front of me one at a time for manual human review.
-
7 · You or I deposit it
I hand you a validated file. You submit it under your own Crossref account, or your press does, or I can. I never hold your credentials and I never deposit on your behalf unless you want me to handle the deposit, too.
The questions editorial teams and boards ask
Where is AI used in this?
In writing the scripts, and nowhere else. I prompt Claude Code to help me write the Python that automates each stage, and once those scripts exist they run on their own. The libraries doing the work on your text — GROBID, AnyStyle, Tesseract — are machine learning in the technical sense, but they're labeling technologies rather than generative ones, and GROBID runs locally in a container on my machine rather than as a call to anybody's API. There is one opt-in path that would use a language model for extraction if every other method failed on a document. Across six journals I've never needed it, and I don't want to, since API calls would also cost me money.
How accurate is it?
Two numbers get confused here, and they measure different things. The first is whether a reference came out of your article intact — the right authors, the right title, one entry rather than two glued together. That number is high, and every reference gets deposited verbatim whether or not it matches anything else, so your works cited land in the record either way. The second is the match rate: what share of those references point at a work that already carries a DOI of its own. Across the six journals now live that one is about 47 percent, 14,680 of 31,053. Some I matched at deposit; Crossref runs its own matcher over the rest and keeps working on what's left. The match rate looks low at first glance, especially next to what the big STEM citation databases report, but for a field in the humanities it's actually a pretty solid number. We cite books, edited collections, dissertations, conference papers, institutional reports, archival material, and a good deal of gray literature that was never going to have a DOI in the first place. The journals that score lower on this measure are not worse-curated or less rigorous on this measure. Reflections matches lowest, at 28 percent, because a journal of community-engaged writing cites community reports, local organizations, and gray literature that was never DOI-eligible. As you might expect, the genre of citation predicts this number far better than rigor does.
What happens to the references that don't match?
They get deposited anyway. Every reference goes in as verbatim citation text along with whatever structured fields could be parsed out of it, matched or not, so the works cited are in the record either way. They also stay matchable: as Crossref's own coverage grows, references deposited today can resolve tomorrow. This is useful as the backlinking process is automated by CrossRef and enhances the citational record even postscript to the work I'll do for your journal.
Who touches our Crossref login?
Nobody but you, unless you ask me to do the deposit for you. Depositor credentials belong to your journal or your press.
Can our authors opt out of getting a DOI? Can they opt out of Pinakes?
Two different questions live inside this one and they're worth separating. Whether an author's name appears in the Pinakes index is an author-level decision, and there's a working tool that honors it. Whether your journal deposits reference lists at all is an editorial decision that belongs to your editorial team or board.
Since you're curious about this question, here is my view: The point of registering a DOI and depositing the citations attached to it is to make an open, shareable record of scholarship so that more scholarship can be built on it. Declining to participate means, in practice, choosing to make your journal's work harder to find and harder to follow. That may be what a given forum wants, and it's a defensible position depending on your particular rhetorical situation. Also of note: publishing open access already entails visibility to the open web, with everything that follows from it, and authors accept that relation when they publish with you.
What does this cost us in staff time?
Very little. A full pipeline run over a complete archive has taken between four and a half minutes and just under four hours depending on the size of the catalog – this is my time to offer to help you move into richly structured metadata practices for your journal. On your side, the back catalog costs you the conversations we have and the time to submit one file. Going forward, depositing a new issue at publication runs about twenty minutes, whether you do it or I do.
Set that against the alternative. Hand-keying the 1,185 articles I've done so far for minting DOIs, at an average of twenty-six references each, is at least a month or two of full-time work, and somebody would have to be paid for it.
Is it ethical to scrape our site?
I think so, under conditions I've tried to state publicly. I've written up nine principles for this kind of collection in a research note forthcoming in The Journal of Writing Analytics, grounded in the Association of Internet Researchers' guidelines and in McKee and Porter's case-based approach. Three of them govern what I'd do to your archive: robots.txt is non-negotiable and a desire for completeness doesn't override consent; requests get rate-limited so collection never burdens infrastructure that a volunteer staff maintains; and where a barrier exists, I document the gap rather than engineer around it.
I'd also note the narrower thing that makes this case different from most scraping debates. Nothing here is human-subjects data. What gets collected is bibliographic metadata your journal already publishes openly, and the full text of the scholarship stays out of bounds except as the source the reference list is read from.
Why should we trust a one-person project?
Check rather than take my word for it. I am Design and Layout Editor at Literacy in Composition Studies, and LiCS is where I ran this first before I offered it to anybody else's. The tools are open source and you or someone on your editorial team or board can read them (github.com/justalewis — the auditor, the Janeway plugin, and Pinakes itself). The deliverable is a standard Crossref file that belongs to your journal.
Where do the numbers in the table come from?
One pair of queries per journal against Crossref's public REST API, run nightly. The total is
api.crossref.org/works?filter=issn:YOUR-ISSN&rows=0, and the count carrying
references is the same query with ,has-references:true added to the filter. Read
total-results off each; the second divided by the first is the Reference lists
percentage. You can run either one in a browser right now. The DOI percentage measures those
registrations against the article count Pinakes has
indexed for the journal.
If your editorial team or board wants to take a next step
Send me a link to your archive and I'll tell you what the job looks like for your journal specifically. That costs you nothing and commits you to nothing.
Or write to me directly at jlewis2@olympic.edu.