Where my life experiences are shared

To see if nodes in your Hadoop cluster connected and run, followings are the ways to see JPS Command Run jps command in master node and slave nodes. In master node, the output of the jps should be something like this $ jps 1634 JobHistoryServer 692 NameNode 2471 Jps 1191 ResourceManager 841 DataNode 1023 SecondaryNameNode 1311 NodeManager When you run jps in slave, the output should be something like this $ jps 528 Jps 292 NodeManager 170 DataNode hdfs dfsadmin -report This is a very nice tool to see the statistic of each nodes in your Hadoop cluster. When you run the command, you will get ouput something like $ hdfs dfsadmin -report Configured Capacity: 82999410688 (77.30 GB) Present Capacity: 59326111744 (55.25 GB) DFS Remaining: 59325284352 (55.25 GB) DFS Used: 827392 (808 KB) DFS Used%: 0.00% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 Pending deletion blocks: 0 ...

If you did not shut down the previous running mininet/containernet network (e.g. if you lose your connection to remote server), you will got the following error when you try to rerun your mininet network Traceback (most recent call last): File "./mynet.py", line 31, in <module> net.addLink(d2, s1) File "build/bdist.linux-x86_64/egg/mininet/net.py", line 403, in addLink File "build/bdist.linux-x86_64/egg/mininet/link.py", line 430, in __init__ File "build/bdist.linux-x86_64/egg/mininet/link.py", line 474, in makeIntfPair File "build/bdist.linux-x86_64/egg/mininet/util.py", line 202, in makeIntfPair Exception: Error creating interface pair (d2-eth0,s1-eth2): RTNETLINK answers: File exists In order to solve the problem, you need to clean up the previous running topology by using the following command sudo mn -c It will clean up all your cache. It will be something like this $ sudo mn -c *** Re...

By default, you have to run docker using sudo . When you try to run it without sudo, you will get $ docker images Got permission denied while trying to connect to the Docker daemon socket In order to run without sudo, you will need to do the following steps Add the docker group if it does not already exist sudo groupadd docker Add the user to the docker group. If you want to add the current user, you can just use this sudo gpasswd -a $USER docker If you want to add, say user user1 , then you should just do sudo gpasswd -a user1 docker After you do the above steps, log out and now you can run docker without sudo .

sendmail In this tutorial, we will write python code to send to multiple emails. Followings are the scenario: Emails will be listed in a text file The Python code will read the emails and send email to each address The email used is gmail Allow Less Secure App You must allow low secure app in your google account setting before you can run the code. In order to do that, go to https://myaccount.google.com/?pli=1 , navigate to sign in and security , scroll down and enable the allow less secure apps Write the Python Code Now we go to the most interesting part i.e. writing the code. Simplest Code We will start with a very simple code to send to one account #!/usr/bin/python import smtplib fromaddr = "mymail@gmail.com" frompass = "mypass" text = "this is the body" email = "targetmail@gmail.com" # login and send the email server = smtplib.SMTP('smtp.gmail.com', 587) server.starttls() server.login(from...