Health Related News Dataset

About the dataset The information collected by Fontes_2023 was organized into a database containing 154,407 news articles with metadata regarding publication date, source (website), news title, headline, and full content. All data manipulation processes were conducted using Jupyter Notebook, with the Polars library version 1.32.0, and the Python programming language version 3.10.12. To identify health-related news articles with potential indications of audit relevance, a three-stage keyword-based selection strategy was applied. In the first stage, across the entire dataset, articles containing the keyword "saúde" (health) in their content (corpus) were classified as "Health" while all others were classified as "Generic News." The generic articles were excluded from subsequent stages. In the second stage, keywords indicating potential irregularities were applied to the subset of articles previously classified as "Health," dividing them into "Generic Health" and "Health with Irregularities." Articles containing at least one of the keywords in the title and/or headline were classified as "Irregularity News." At this stage, 6,239 articles were identified as containing indications of irregularity, while the remaining 3,277 were classified as "Generic Health." The following keywords are used in these stage: "abuso, abuso de poder, acordo ilegal, acusaç, acusação, apropriação indébita, auditoria, aumento orçamento, bilhões, cartel, coação, compra, compras públicas, conluio, contrato, contratos, corrupto, corrupção, corte orçamento, crime, crime organizado, criminoso, deflagrou, denuncia, denúncia, desassistência, desfalque, desonestidade, desonesto, desperdício, desvio, desvios, disfarce, documento alterado, dolo, enganar, engano, enganos, enganoso, enriquecimento, enriquecimento ilícito, escândalo, esquema, evasão, falcatrua, falsa declaração, falsificado, falsificador, falsificação, falso, falta, falta de equipamentos, falta equipamento, fiscalização, forjar, fraudador, fraudar, fraude, fraude em contratos, fraude em licitações, fraude financeira, fraude licitação, fraudulento, fugir, golpe, ilegal, ilusão, ilícito, indicativo, indício, infração, investiga, investigação, irregular, irregularidade, irregularidade administrativa, irregularidade de gestão, irregularidade financeira, irregularidades, lavagem, lavagem de dinheiro, licitação, mandado, manipulado, manipulador, manipulação, manipulação de dados, maquiar, mentira, milhões, má conduta, negligência dolosa, ocultar, ocultação, PF, peculato, perjúrio, plano, plano de saúde, plano saúde, prevaricação, propina, recurso, relatório falso, rombo, roubo, sem autorização, sem consentimento, sobrefaturamento, sonegar, sonegação, suborno, sugestão, superfaturamento, suspeit, suspeita, suspeito, transação, transação suspeita, transgressão, transparência, uso indevido, uso indevido de recursos, plano saude, uso irregular, venda." In the third stage, articles belonging to the "Irregularity News" subgroup were independently assessed by two annotators. A substantial inter-rater agreement (IRA) was achieved, with a Cohen’s Kappa ($k$) value of 0.6203. Five additional evaluators were assigned to resolve cases of disagreement in the classification. Each evaluator was individually responsible for making a final decision on 65 distinct samples. The assessment involved reviewing the titles and abstracts of all articles to confirm their categorization as "Health Irregularity," "Generic Health," or "Generic News." Finally, after the third stage, 421 health-related articles with indications of irregularity were identified. Figure below illustrates the news selection process. For the use of this dataset in the text summarization task, it was necessary to create summaries of the original articles to serve as reference summaries for the evaluation of the automatically generated ones. These reference summaries were produced by an external journalist researcher. Files df_health_news_raw: Contains the raw information. df_health_news_filtred: Contains the final dataset, after the process of noting news items with evidence of irregularities and creating summaries.