Toward Automated Worldwide Monitoring of Network-level Censorship

Although Internet censorship is a well-studied topic, to date most published studies have focusedon a single aspect of the phenomenon, using methods and sources specific to each researcher.Results are dicult to compare, and global, historical perspectives are rare. Because each groupmaintains their own software, erroneous methods may continue to be used long after the error hasbeen discovered. Because censors continually update their equipment and blacklists, it may beimpossible to reproduce historical results even with the same vantage points and testing software.Because “probe lists” of potentially censored material are labor-intensive to compile, requiring anunderstanding of the politics and culture of each country studied, researchers discover only the mostobvious and long-lasting cases of censorship.In this dissertation I will show that it is possible to make progress toward addressing all ofthese problems at once. I will present a proof-of concept monitoring system designed to operatecontinuously, in as many dierent countries as possible, using the best known techniques fordetection and analysis. I will also demonstrate improved techniques for verifying the geographiclocation of a monitoring vantage point; for distinguishing innocuous network problems fromcensorship and other malicious network interference; and for discovering new web pages that areclosely related to known-censored pages. These techniques improve the accuracy of a continuousmonitoring system and reduce the manual labor required to operate it.This research has, in addition, already led to new discoveries. For example, I have confirmedreports that a commonly-used heuristic is too sensitive and will mischaracterize a wide variety ofunrelated problems as censorship. I have been able to identify a few cases of political censorshipwithin a much longer list of cases of moralizing censorship. I have expanded small seed groups ofpolitically sensitive documents into larger groups of documents to test for censorship. Finally, Ican also detect other forms of network interference with a totalitarian motive, such as injection ofsurveillance scripts.In summary, this work demonstrates that mostly-automated measurements of Internet censorshipon a worldwide scale are feasible, and that the elusive global and historical perspective is withinreach.