Extraction of function words.

In this paper, our investigation uses up to 100 function words for the English corpus, the Federalist Papers, and the Koine Greek texts. The frequencies of function words in each text are used as classification features. The texts are initially pre-processed as follows: (i) all letters are changed to lower case, (ii) all accents are removed, (iii) all ASCII characters not in the set a-z (ASCII codes 97–122) and space (ASCII code 32) are removed without insertion of a space. This is with the exception of a hyphen (ASCII code 45) that is substituted with a space (ASCII code 32), (iv) all headings are removed from the texts, so that they only contain free flowing paragraphs, (v) any extra items added by modern editors, such as editorial notes are removed. Whilst hagiographers wrote the Greek texts in capital letters without accents or punctuation, modern editors insert these items for ease of interpretation. Thus, in order to recover the original Greek text, we apply steps (i) to (v). We do the same to English texts so that they act as a punctuation-free benchmark test. As our software only handles the reduced 27-character ASCII set a-z and space, the Greek text is transliterated using the Table 8. After this pre-processing, all the texts within each corpus are concatenated and word frequencies are counted. Words are ranked in descending order of frequency of occurrence. This is shown in the table below. These key words are called function words.