Mannheimer Webpanel

The ZEW-FDZ offers a novel panel of semi-structured webpage data on company level – the Mannheimer Webpanel. It comprises textual webpage data retrieved from a broad range of German firm websites. A detailed description of the webscraping methods used to harvest the data as well as an examination of the dataset (corpus of German corporate websites) can be found in this discussion paper. The dataset provides, among others, the following variables:<br>- ID – unique company identifier.<br>- dl_rank – usually a company website consists of several single webpages. In this context, dl_rank represents the chronological order in which the individual webpages were downloaded. The main page of a website has rank 0, the first subpage processed after the main page has rank 1, and so on.<br>- dl_slot – the domain name of the website.<br>- title – the title of the company website as indicated in the website's meta data.<br>- keywords – list of keywords of the company website as indicated in the website's meta data.<br>- description – the description of the company website as indicated in the website's meta data.<br>- text – the text/content that was downloaded from the webpage.<br>- timestamp – the exact time when the webpage was downloaded.<br>- url – the URL of the webpage.