Replication data and code for "The .GOV Internet Archive: A Big Data Resource for Political Science"

Code for Scraping the Altiscale/Internet Archive .gov Data for Political Science Research Documents in this repository include scripts for scraping the .gov data for two main purposes: 1) tracing the history of an entire webpage and either outputting the full text of those pages or outputting keywords from those pages 2) tracing the entire history of a given set of terms, and then either outputting the full text of those pages or outputting keyword counts from those pages a. also possible to sort and categorize by URL group (the URL roots of interest)