SM01: Web Crawling with JLC parallel execution pattern - experiment reports

Research project SM01 (Parallel Semantic Crawler for manufacturing multilingual web...) The JLC pattern is running parallel thread for each domain in the Crawl Job domain cue. Objective of the experiment was to find what number of parallel DLC threads (TCmax) results in the highest execution efficiency. The crawlers were run: 2, 4, 6, 8, 10, 12, 14 and 16 parallel DLC threads, against the Sc subset. Other configuration parameters: Load Take per iteration (LT) set to 1 and Page Loads limit (PLmax) set to 30. Sampling period of the resource utilization: 5 seconds.