Using PubCrawler: How to Speed Up Your Literature Searches by Crawling

It often seems like a full-time job searching PubMed for the latest papers on adult neurogenesis, mutations in kinase genes that lead to cancer, or the newest papers published from a major figure in your field. However, falling behind on the latest published research can also be a significant source of stress.

In search of a library assistant

When I was a graduate student, I would diligently search PubMed for the newest articles published on my then favorite protein, TrkB, and my favorite scientist, Marc Tessier-Lavigne. However, given the demands of graduate school in the experimental sciences, it can be difficult to keep up with your favorite sub-searches on a regular basis. “If only something existed that would do these periodic searches for me automatically and simply email them to me,” I thought.

Turns out, it existed.

PubCrawler is a free, web-based service developed by Karsten Hokamp and Ken Wolfe in the Department of Genetics at Trinity College Dublin, Ireland. PubCrawler automatically searches both PubMed and GenBank for text-based queries that you specify, and emails you the results. After signing up for an account, you can customize search queries in several ways, including those offered by PubMed (author, title words, text/abstract keywords, MESH terms, journal, publication range, etc.). You also have the options of selecting how frequently your search results are emailed to you and which specific day of the week (and time!) you’d like to receive them.

Though PubCrawler has existed since 1999, it is still an absolutely vital part of me staying on top of the literature in my field.

How to use PubCrawler to go to the library for you

For example, let’s say that you want to keep abreast of the latest published work from Fred Gage, a prominent player in the field of adult neurogenesis. Simply construct a new query using “Gage FH” as the search term and using the drop-down menu under search field, select “author”. Hit the finish button at the bottom and you’ll now receive emails whenever Fred Gage publishes a new paper according to the frequency and time you’ve set:

Figure 1 pubcrawler

Figure 2 pubcrawler

The search queries you can set up with PubCrawler are as diverse as those possible searching PubMed directly. For example, you might also want to know when papers are published on the role of the NMDA receptor in memory. For this, simply use “NMDA receptor” as the search term in one box and “memory” as the search terms below.

Figure 3 pubcrawler

Select “and” as the connector to make sure that it returns papers that have both of these terms (in this example, eligible returns must have each term in the title or abstract of the paper, but you can restrict it to just the title or broaden it to retrieve papers that have the chosen terms anywhere in the text). By using combinations of search terms, you can create custom queries to deliver only the papers you want!

Figure 4 pubcrawler

A more thorough tutorial can be found here.

If you agree with PubCrawler’s slogan (“It goes to the library – you go to the pub”), then setting up a series of queries is an absolutely vital part of staying on top of the literature in your field in the most efficient way!

1 Comment

  1. Mark Fortner on February 2, 2016 at 5:57 am

    Why wouldn’t you just use myncbi for that. It has the same functionality and lets you get scheduled results as email or RSS.

Leave a Comment