Posts

Showing posts from July, 2018

Format Excel MailMerge Date fields

MERGEFIELD  Form_Deadline  \@ dd-MMM-yyyy

Useful Raspbian commands

Check Raspbian version cat / etc / os - release

Google Chart - Add percentage to PieChart

https://stackoverflow.com/questions/19178029/want-to-show-percentage-data-in-legends-of-google-pie-chart 8 down vote accepted You can do this two ways; either set the formatted values of the labels in the DataTable to include the percents, eg: // assume sleep is row 4, "sleepPercent" is the percent for sleep data . setFormattedValue ( 4 , 0 , data . getValue ( 4 , 0 ) + ' (' + ( sleepPercent * 100 ). toFixed ( 1 ) + '%)' ); or you can set the  legend.position  option to "labeled", which changes the way the legend is drawn, but adds the percents to the labels: legend : { position : 'labeled' } see an example here:  http://jsfiddle.net/asgallant/Su6TX/

Sort a Google DataTable

https://stackoverflow.com/questions/23764280/sort-in-google-table-chart Sort your DataTable: // sort by column 0, descending data . sort ({ column : 0 , desc : true }); or use a sorted DataView: var view = new google . visualization . DataView ( data ); view . setRows ( data . getSortedRows ({ column : 0 , desc : true })); table . draw ( view , options );

Format a multi key-value MQTT message in Python

Here's the code to formulate a multi-attribute (multi key-value) MQTT message in Python mqtt_message = "{\"value\":\"" + str(light) + "\",\"serialNumber\":\"" + str(loopCount) + "\",\"sensorType\":\"light\"}"

Cloudera CDH

Hue https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_command-line-installation/content/start_stop_restart_hue.html Install Docker Cloudera 5.13 https://gist.github.com/davideicardi/21aabb65faaa3c655770cf0ccbac6564

Docker useful commands

Listing containers https://docs.docker.com/engine/reference/commandline/container_ls/ docker container ls -all Remove containers https://docs.docker.com/engine/reference/commandline/rm/ docker rm quickstart.cloudera Stopping containers docker stop quickstart.cloudera  # Stop container ( link ) docker container prune # Remove all stopped containers docker volume prune # Remove all unused volumes docker image prune # Remove unused images Links https://hackernoon.com/clean-out-your-docker-images-containers-and-volumes-with-single-commands-b8e38253c271 https://stackoverflow.com/questions/44480740/how-to-save-a-docker-container-state https://community.cloudera.com/t5/Web-UI-Hue-Beeswax/The-application-won-t-work-without-a-running-HiveServer2/td-p/30627

MongoDB + Python CRUD examples

This is for my PDS class Retrieve records import pymongo from pymongo import MongoClient client = MongoClient() # Get the sampleDB database db = client.adventureworks collection = db.inventory # Get all items in the collection  for item in collection.find():     print(item) # Get all items in the collection with cost less than $1.00 for item in collection.find({"cost": {"$lt":1}}):     print(item) Insert records ### EXAMPLE to show how you insert documents into MongoDB import pymongo from pymongo import MongoClient client = MongoClient() # Get the sampleDB database db = client.adventureworks collection = db.inventory # Insert three documents collection.insert_many( [     {"name": "Protractor", "cost": 0.50, "color": "red"},     {"name": "Pencil case", "cost": 6.50, "type": "zipper", "d

Load data from CSV and plot using Seaborn barplot method

Image
For my PDS class! %matplotlib inline import mysql.connector from datetime import date, datetime import sys import pandas as pd import seaborn as sns user,pw, host,db = 'root','it8701','127.0.0.1','anotherdatabase2' cnx = mysql.connector.connect(user=user, password=pw, host=host, database=db) cursor = cnx.cursor() select_stmt = ("SELECT * FROM cea_salespersons") try:   cursor.execute(select_stmt)   df = pd.DataFrame(cursor.fetchall(), columns = ['cea_salesperson_id','salesperson_name', 'registration_no',                                                   'registration_start_date',  'registration_end_date',                                                    'estate_agent_name','estate_agent_license_no'])        df["registration_start_date"] = pd.to_datetime(df["registration_start_date"], errors="coerce")   # Create a new DataFrameGroup

Plot a bar graph using DataFrame

Image
Untitled Document This is to answer a student's question on WhatsApp %matplotlib inline import pandas as pd import calendar #data is from https://data.gov.sg/dataset/rainfall-monthly-total df = pd.read_csv("data/rainfall-monthly-total.csv") # make the month column a datetime type df.month = pd.to_datetime(df.month, errors='coerce') # Create a new column that just has the year df['date_year'] = df.month.dt.year # Create a new column that just has the month df['date_month']= df.month.dt.month # Format the month number as a month name df['date_monthname']= df['date_month'].apply(lambda x: calendar.month_abbr[x]) # Extract only rainfall data for 2017 df_2017 = df[df.date_year==2017] # Extract only two columns, the monthname and total rainfall df_2017 = df_2017[["date_monthname", "total_rainfall"]] # Set index to be the monthname, that will auto format the x-ticks as the month names df_2017.set_in

Plotting a bar plot with DataFrame

Image
For my class demo on 12 Jul 2018 ## Concat %matplotlib inline import pandas as pd import matplotlib.ticker as ticker import matplotlib.pyplot as plt import numpy as np d1 = pd.read_csv("data/ura-private-district9.csv",skiprows=2) d2 = pd.read_csv("data/ura-private-district3.csv",skiprows=2) d = pd.concat([d1,d2]) d = d.rename(index=str, columns={"S/N":"SerialNumber","Price ($)":"Price", "Project Name": "Project"}) d["SerialNumber"] = pd.to_numeric(d["SerialNumber"], errors="coerce") d = d[~pd.isnull(d.Price)] d["Unit Price ($psf)"] = pd.to_numeric(d["Unit Price ($psf)"], errors="coerce") d["Date of Sale"] = pd.to_datetime(d["Date of Sale"], errors="coerce") projects = d.groupby("Project") medianpricebyproject = projects['Price'].median() medianpricebyproject= pd.DataFra