Speedtest Open Data - Australia(NZ) 2020-2024; Q220 - Q424 extract by Qtr

This is an Australian extract of Speedtest Open data available at Amazon WS (link below - opendata.aws).<br>AWS data licence is "CC BY-NC-SA 4.0", so use of this data must be:- non-commercial (NC)- reuse must be share-alike (SA)(add same licence).This restricts the standard CC-BY Figshare licence.<br>A world speedtest open data was dowloaded (&gt;400Mb, 7M lines of data). An extract of Australia's location (lat, long) revealed 88,000 lines of data (attached as csv).<br>A Jupyter notebook of extract process is attached.<br>See Binder version at Github - https://github.com/areff2000/speedtestAU.+&gt; Install: 173 packages | Downgrade: 1 packages | Total download: 432MB<br>Build container time: approx - load time 25secs.<br>=&gt; Error: Timesout - BUT UNABLE TO LOAD GLOBAL DATA FILE (6.6M lines).=&gt; Error: Overflows 8GB RAM container provided with global data file (3GB)=&gt; On local JupyterLab M2 MBP; loads in 6 mins.<br>Added Binder from ARDC service: https://binderhub.rc.nectar.org.auDocs: https://ardc.edu.au/resource/fair-for-jupyter-notebooks-a-practical-guide/<br>A link to Twitter thread of outputs provided.A link to Data tutorial provided (GitHub), including Jupyter Notebook to analyse World Speedtest data, selecting one US State.<br>Data Shows: (Q220)- 3.1M speedtests- 762,000 devices- 88,000 grid locations (600m * 600m), summarised as a point- average speed 33.7Mbps (down), 12.4M (up)- Max speed 724Mbps- data is for 600m * 600m grids, showing average speed up/down, number of tests, and number of users (IP). Added centroid, and now lat/long.<br>See tweet of image of centroids also attached.<br>NB: Discrepancy Q2-21, Speedtest Global shows June AU average speedtest at 80Mbps, whereas Q2 mean is 52Mbps (v17; Q1 45Mbps; v14). Dec 20 Speedtest Global has AU at 59Mbps. Could be possible timing difference. Or spatial anonymising masking shaping highest speeds. Else potentially data inconsistent between national average and geospatial detail. Check in upcoming quarters.<br>NextSteps:Histogram - compare Q220, Q121, Q122. per v1.4.ipynb.<br>Versions:v39: Added AUS Q424 (95k lines avg d/l 110.9 Mbps u/l 21.02 Mbps). Imported using v2 Jupyter notebook (MBP 16Gb). Mean tests: 17.2. Mean devices: 5.24. Download, extract and publish: 14 mins.v38: Added AUS Q324 (92k lines avg d/l 107.0 Mbps u/l 20.79 Mbps). Imported using v2 Jupyter notebook (iMac 32Gb). Mean tests: 17.7. Mean devices: 5.33.Added github speedtest-workflow-importv2vis.ipynb Jupyter added datavis code to colour code national map. (per Binder on Github; link below).v37: Added AUS Q224 (91k lines avg d/l 97.40 Mbps u/l 19.88 Mbps). Imported using speedtest-workflow-importv2 jupyter notebook. Mean tests:18.1. Mean devices: 5.4.v36 Load UK data, Q1-23 and compare to AUS and NZ Q123 data. Add compare image (au-nz-ukQ123.png), calc PlayNZUK.ipynb, data load import-UK.ipynb. UK data bit rough and ready as uses rectangle to mark out UK, but includes some EIRE and FR. Indicative only and to be definitively needs geo-clean to exclude neighbouring countries.v35 Load Melb geo-maps of speed quartiles (0-25, 25-50, 50-75, 75-100, 100-). Avg in 2020; 41Mbps. Avg in 2023; 86Mbps. MelbQ323.png, MelbQ320.png. Calc with Speedtest-incHist.ipynb code. Needed to install conda mapclassify. ax=melb.plot(column=...dict(bins[25,50,75,100]))v34 Added AUS Q124 (93k lines avg d/l 87.00 Mbps u/l 18.86 Mbps). Imported using speedtest-workflow-importv2 jupyter notebook. Mean tests:18.3. Mean devices: 5.5.v33 Added AUS Q423 (92k lines avg d/l 82.62 Mbps). Imported using speedtest-workflow-importv2 jupyter notebook. Mean tests:18.0. Mean devices: 5.6. Added link to Github.v32 Recalc Au vs NZ for upload performance; added image. using PlayNZ Jupyter. NZ approx 40% locations at or above 100Mbps. Aus &lt;5%, perhaps &lt;2%.v31 Added graph of NZ vs Aus Q3 2023 Broadband performance; and PlanNZ Jupyter notebook.v30 Added NZ Q323 ( 20k lines avg d/l 154.33Mbps). Mean tests: 10.3 Mean devices: 3.4.<br>nz_tiles = tiles.cx[166.509144322:178.517093541 , -46.641235447:-34.4506617165]<br>Source: https://gist.github.com/graydon/11198540v29 Added AUS Q323 (90k lines avg d/l 79.08 Mbps). Imported using speedtest-workflow-importv2 jupyter notebook. Mean tests:16.1. Mean devices: 5.2.v28 Added v2 Import ipynb with new histograms to examine quarterly data.V27.1 Added AUS Q223 (90k lines avg d/l 74.25 Mbps). Imported using speedtest-workflow-import jupyter notebook. Mean tests:15.7. Mean devices: 5.1.V27 Added AUS Q123 (89k lines avg d/l 69.69 Mbps). Imported using speedtest-workflow-import jupyter notebook. Mean tests:16. Mean devices: 5.3.V26 Add DOI to Journal Publication (linked below). International Regional Science Review.v25 Added AUS Q422 geojson (89k lines, avg d/l 65.9 Mbps). Imported using speedtest-workflow-import jupyter notebook. Mean tests:19. Mean devices: 5.8.v24 Added AUS Q322 geojson (83k lines, avg d/l 61.6 Mbps), plus Workflow-import jupyter notebook (speedtest-workflow-import).v23 Added AUS Q222 geojson (87k lines, avg d/l 58.4 Mbps)v22 Added AUS Q122 geojson (88k lines, avg d/l 56.9 Mbps). Mean tests: 21. Mean devices: 6.4.v21: Added AUS Q421 geojson (86k lines, avg d/l 57.6 Mbps). Mean tests: 21. Mean devices: 6.5.v20: Added AUS Q321 geojson. (81k lines avg d/l 54.9 Mbps). Mean tests: 27. Mean devices: 7.v18/19: Add linechart, Histogram and v1.4 ipynb, comparing Q221 to Q121 and Q220. Speedtest Global at 0621 puts AU at 80Mbps average (June). Substantially above data mean (Apr, May Jun) (v17).v17: Add AUS Speedtest Q2 2021 geojson.(79k lines avg d/l 52.3Mbps)v15/16. Add Hist comparing Q1-21 vs Q2-20. Inc ipynb (incHistQ121, v.1.3-Q121) to calc.v14 Add AUS Speedtest Q1 2021 geojson.(79k lines avg d/l 45.4Mbps)v13 - Added three colour MELB map (less than 20Mbps, over 90Mbps, 20-90Mbps)v12 - Added AUS - Syd - Mel Line Chart Q320.v11 - Add line chart compare Q2, Q3, Q4 plus Melb - result virtually indistinguishable. Add line chart to compare Syd - Melb Q3. Also virtually indistinguishable. Add HIST compare Syd - Melb Q3. Add new Jupyter with graph calcs (nbn-AUS-v1.3). Some ERRATA document in Notebook. Issue with resorting table, and graphing only part of table. Not an issue if all lines of table graphed.v10 - Load AURIN sample pics. Speedtest data loaded to AURIN geo-analytic platform; requires edu.au login.v9 - Add comparative Q2, Q3, Q4 Hist pic.v8 - Added Q4 data geojson. Add Q3, Q4 Hist pic.v7 - Rename to include Q2, Q3 in Title.v6 - Add Q3 20 data. Rename geojson AUS data as Q2. Add comparative Histogram. Calc in International.ipynb.v5 - add Jupyter Notebook inc Histograms. Hist is count of geo-locations avg download speed (unweighted by tests).v4 - added Melb choropleth (png 50Mpix) inc legend. (To do - add Melb.geojson). Posted Link to AURIN description of Speedtest data.v3 - Add super fast data (&gt;100Mbps) less than 1% of data - 697 lines. Includes png of superfast.plot(). Link below to Google Maps version of superfast data points. Also Google map of first 100 data points - sample data. Geojson format for loading into GeoPandas, per Jupyter Notebook. New version of Jupyter Notebook, v.1.1.v2 - add centroids image.v1 - initial data load.<br>** Future Work- combine Speedtest data with NBN Technology by location data (national map.gov.au); https://www.data.gov.au/dataset/national-broadband-network-connections-by-technology-type- combine Speedtest data with SEIFA data - socioeconomic categories - to discuss with AURIN.- Further international comparisons- discussed collaboration with Assoc Prof Tooran Alizadeh, USyd. - <i>"It is a pleasure to accept your manuscript entitled "The Multi-Technology Footprint of the National Broadband Network in Australia: Exploring the urban-regional divide and socio-spatial patterns for inequality" in its current form for publication in </i>International Regional Science Review<i>." (1.3.23) DOI now added in links below (https://doi.org/10.1177/01600176231168025).</i>