Posts

Embed JSX code with if condition in React app

     renderPrice(pt)  {    if (pt.amountpaid!="") {        let ctrl = (           <>             <div>                 <label>Price</label><br/>                 <input type="text" readOnly name="amountpaid" value={pt.amountpaid} />             </div>           </>          );          return ctrl;     } //end if } //end renderPrice render () {   const   priceElement  =  this . renderPrice ( purchasetransaction ); return  (                  < div   style = { { marginTop :   '5em' , marginLeft : '2em' } } >  ...

Reset root password on MySQL on Windows 10 laptop

Stop SQL Server Create a text file "mysql-init.txt" in C:\temp directory and paste in this content ALTER USER 'root'@'localhost' IDENTIFIED BY 'MyNewPass'; Open a command prompt window and run this command mysqld --defaults-file="C:\\ProgramData\\MySQL\\MySQL Server 8.0\\my.ini" --init-file="C:\\temp\\mysql-init.txt"

Create a SQL table with autoincrement id and timestamps

CREATE TABLE `tutorials` (   `id` int(11) NOT NULL AUTO_INCREMENT,   `title` varchar(255) DEFAULT NULL,   `description` varchar(255) DEFAULT NULL,   `published` tinyint(1) DEFAULT '1',   `createdAt` timestamp NULL DEFAULT CURRENT_TIMESTAMP,   `updatedAt` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,   PRIMARY KEY (`id`) ) 

AWS Policy editor

https://awspolicygen.s3.amazonaws.com/policygen.html

Download files from Databricks Community Edition

Syntax is: https://community.cloud.databricks.com/files/myStuffs/yourdata.csv/part-00000?o=<yournumber>

Run pyspark on Juypter notebook

$which python $whereis python # install EPEL repository first $ sudo yum install epel-release # install python-pip $ sudo yum -y install python-pip sudo pip install --upgrade setuptools wget https://repo.anaconda.com/archive/Anaconda2-5.0.1-Linux-x86_64.sh sudo sh Anaconda2-5.0.1-Linux-x86_64.sh 1) Install PySpark pip install pyspark 2) Install Java 3) Install Jupyter notebook pip install jupyter 4) Install find pip install findspark %env SPARK_HOME=c:\spark # To find out where the pyspark import findspark findspark.init() # Creating Spark Context from pyspark import SparkContext sc = SparkContext("local", "first app") # Calculating words count text_file = sc.textFile("OneSentence.txt") counts = text_file.flatMap(lambda line: line.split(" ")) \              .map(lambda word: (word, 1)) \              .reduceByKey(lambda a, b: a + b) # Printing each word with its respective coun...

Useful icons

https://pngtree.com/free-icon/node-other-database-cluster_741016 https://www.clipartmax.com/

Set up Cloudera CDH 5.13 todo

sudo /etc/resolv.conf Edit Virtual Box forwarding, add port 22

Troubleshooting Hive

https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cdh_ig_hiveserver2_start_stop.html Check if Hive metastore is running sudo service hive-metastore status sudo service hive-metastore stop sudo service hive-metastore start sudo service hive-metastore restart sudo service hive-server2 stop Check if Hive Server2 is running $ sudo service hive-server2 status Start HiveServer2: $ sudo service hive-server2 start To stop HiveServer2: $ sudo service hive-server2 stop Check Hive warehouse hadoop fs -ls /user/hive/warehouse Connect using Beeline beeline -u jdbc:hive2:// !connect -u jdbc:hive2://

Login Hortonworks Sandbox with WinSCP

ssh root@sandbox-hdp.hortonworks.com -p 2222 First-time login, use hadoop as password You will be asked to change password upon first login C:\Users\<username>\.ssh

kill yum process

yum list ps -ef | grep 13023 kill -9 13023

Troubleshoot DNS on CDH

sudo nano /etc/resolv.conf nameserver 8.8.8.8 # /etc/init.d/network stop # /etc/init.d/network start sudo systemctl restart network https://www.cyberciti.biz/faq/linux-restart-network-interface/

Download zip file using Python

https://www.zerotosingularity.com/blog/downloading-datasets-introducting-pdl/ import requests import zipfile import os # download file resp = requests.get(f"{url}{file}", allow_redirects=True, stream=True) filename = f"{download_dir}{file}" zfile = open(filename, 'wb') zfile.write(resp.content) zfile.close() zipf = zipfile.ZipFile(filename, 'r') zipf.extractall(download_dir) zipf.close() os.remove(filename) YEAR=2015 for MONTH=`seq -w 1 12`; do PARAMS="UserTableName... RESPONSE=$(curl -X POST --data "$PARAMS" http://www.transtats.bts.gov/DownLoad_Table.asp?Table_ID=236&Has_Group=3&Is_ Zipped=0) echo "Received $RESPONSE" ZIPFILE=$(echo $RESPONSE | tr '\"' '\n' | grep zip) echo $ZIPFILE curl -o $YEAR$MONTH.zip $ZIPFILE done Is this useful? https://stackoverflow.com/questions/51665087/python-download-zip-file-from-restapi

Get a book from Gutenberg

wget http://www.gutenberg.org/cache/epub/11/pg11.txt

utorial: Using pandas with Large Data Sets

https://www.dataquest.io/blog/pandas-big-data/

Adjust width of pandas dataframe

https://stackoverflow.com/questions/52369572/python-how-to-get-data-types-for-all-columns-in-csv-file pd . set_option ( 'max_info_columns' , 200 )

Create a Python 3.6 environment on top of Python 3.7

cd d:\ python -m venv D:\mypython36_venv rem cd d:\mypython36_venv D:\mypython36_venv\Scripts\activate D:\mypython36_venv\Scripts\deactivate This one is for Python 2.7 on Raspberry Pi python -m virtualenv python_venv cd python_venv source mypython/bin/activate

Find out where is mysql data directory

'SHOW VARIABLES WHERE Variable_Name LIKE "%dir" 'SHOW VARIABLES WHERE Variable_Name = "datadir"'

datasets

https://www.chicago.gov/city/en/depts/dhr/dataset/current_employeenamessalariesandpositiontitles.html Sourced from: https://janakiev.com/blog/pandas-groupby/

check if ambari is ready

Image
https://ambari.apache.org/1.2.0/installing-hadoop-using-ambari/content/ambari-chap2-3.html\ 3. Start the Ambari Server To start the Ambari Server: ambari-server start To check the Ambari Server processes: ps -ef | grep Ambari To stop the Ambari Server: ambari-server stop https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Shell