Dora the Techplorer

https://www.zerotosingularity.com/blog/downloading-datasets-introducting-pdl/ import requests import zipfile import os # download file resp = requests.get(f"{url}{file}", allow_redirects=True, stream=True) filename = f"{download_dir}{file}" zfile = open(filename, 'wb') zfile.write(resp.content) zfile.close() zipf = zipfile.ZipFile(filename, 'r') zipf.extractall(download_dir) zipf.close() os.remove(filename) YEAR=2015 for MONTH=`seq -w 1 12`; do PARAMS="UserTableName... RESPONSE=$(curl -X POST --data "$PARAMS" http://www.transtats.bts.gov/DownLoad_Table.asp?Table_ID=236&Has_Group=3&Is_ Zipped=0) echo "Received $RESPONSE" ZIPFILE=$(echo $RESPONSE | tr '\"' '\n' | grep zip) echo $ZIPFILE curl -o $YEAR$MONTH.zip $ZIPFILE done Is this useful? https://stackoverflow.com/questions/51665087/python-download-zip-file-from-restapi

wget http://www.gutenberg.org/cache/epub/11/pg11.txt

https://www.dataquest.io/blog/pandas-big-data/

https://stackoverflow.com/questions/52369572/python-how-to-get-data-types-for-all-columns-in-csv-file pd . set_option ( 'max_info_columns' , 200 )

cd d:\ python -m venv D:\mypython36_venv rem cd d:\mypython36_venv D:\mypython36_venv\Scripts\activate D:\mypython36_venv\Scripts\deactivate This one is for Python 2.7 on Raspberry Pi python -m virtualenv python_venv cd python_venv source mypython/bin/activate

'SHOW VARIABLES WHERE Variable_Name LIKE "%dir" 'SHOW VARIABLES WHERE Variable_Name = "datadir"'

https://www.chicago.gov/city/en/depts/dhr/dataset/current_employeenamessalariesandpositiontitles.html Sourced from: https://janakiev.com/blog/pandas-groupby/