Data from a article "Analise dos fatores econômicos, climáticos e científico sobre a produção de leite brasileira e suas diferenças regionais"

RBQL.csv CSV file presenting the names of cities and their respective latitude and longitude information for laboratories accredited by the Brazilian Milk Quality Network. embrapa.csv CSV file presenting the names of cities and their respective latitude and longitude information for Embrapa units scattered across Brazil. Dataset_final.csv The CSV file has 24 rows (one for each year from 2000 to 2023) and 25 columns (variables). It is a temporal dataset ranging from 2000 to 2023 column (year). Generally, it can be divided into 4 different categories, each possessing data for every region of Brazil: Regional milk production, column (average milk production); Climate data (temperature and precipitation) measured in degrees Celsius and millimeters respectively, columns (Average temperature and Average precipitation); Milk price data, involving average production price data, column (average milk price for the producer) and the average price of milk traded between producers, column (Average net milk price (between producers)). Finally, there is inflation variation data, column (ipca_variacao). quality_data.csv The CSV file contains a total of 4 Parameters CPP, CSS, EST, and ESD, grouped by state/region and by year from 2013 to 2023. The values presented are the maximum and minimum values of that respective year for each parameter. script_de_processamento_leite.ipynb The file presents the code developed using python notebooks, in which different libraries and processing steps are applied: the main tools used are: pandas and numpy: Data manipulation and aggregation. sklearn (KNNImputer): Used to fill in missing data (imputation), ensuring the final dataset has no gaps. prov: Library specialized in documenting data origin (provenance), generating flow diagrams. The script processes data in thematic blocks that correspond directly to the columns of your final CSV: A. Price Data (Regional) The script reads multiple CSV files from regional folders (South, Southeast, Center-West, North, Northeast). Renames original columns (such as "Minimum gross price") and calculates annual averages. Generates the columns: average milk price... and Average net price.... B. Climate Data (Temperature and Precipitation) Iterates over raw files from meteorological stations (likely from INMET). Extracts temperature and rain data, calculates annual averages/medians for each region. Generates the columns: Average temperature... and Average precipitation... for each region (SOUTH, SOUTHEAST, etc.). C. Production Data Loads a specific productivity file (producao). Generates the columns: average milk production... (millions). D. Economic Data Processes inflation files to calculate the annual variation. Generates the column: ipca_variacao. 3. Data Treatment Part of the script uses the KNN (K-Nearest Neighbors) algorithm to fill in null values. 4. Documentation and Provenance The end of the script (after CSV generation) is dedicated to creating "Data Provenance" diagrams. It visually maps where each piece of data came from (e.g., "INMET" -> "Climate Processing" -> "Final Dataset"), ensuring scientific traceability for the project. leite_visualization_script.ipynb Geospatial Analysis (Maps) The script uses the folium, geopandas, and HTML libraries to create interactive visualizations. Input Data: manually defines a dictionary coordenadas_cidades with latitudes and longitudes of Brazilian capitals (Rio Branco, Maceió, Macapá, etc.). Visualization: Generates maps pointing out locations, correlating Embrapa headquarters or milk quality centers with these coordinates. milk_production._map.html HTML file, which allows access to the constructed map and presents the dataset data dynamically, grouped by region. Furthermore, it presents information related to the geographic position of milk research and analysis centers (Embrapa and RBQL). Prov_xxx.png All files starting with prov_ present the provenance of each obtained datum in image format. The image presents information about the data acquisition and transformation process, from the location where it was obtained until being transformed into the dataframe format to be unified in the final dataset in the future. Specifically, quality_prov.png is the provenance graph of the quality dataset. requirements_script_leite.txt The file requirements_script_leite.txt presents information on the libraries used in the environment where the data processing and dataset construction script was executed. It is worth mentioning that the cited environment was Colab, so the main packages essential to run the file are presented in the produced article. requirements_map_script.txt The file requirements_script_leite.txt presents information on the libraries used in the environment where the script for the development of data visualization was executed.