US DOE Energy Efficiency & Renewable Energy

This is a web scraping of all pages with links, saved as HTML and also text files, along with tabulated lists of pages and their links to two layers deep, ie the initial page with links, the child pages and their links. If a password was required, the download was failed within the R code loop (example: DOE has a facebook page only accessible to those with facebook accounts). The code itself is included in the archive, called 'scraplinkswithtextretrieval.R. along with the 'level2tracker*' files which show the names of the pages and how they are linked together. The 'true' vs False in the name of each page indicates whether or not the html code contains the word 'table'. I was looking for tabulated data and did not find much of anything that was not small enough to export as part of the html pages themselves. <br>