Skip to main content

Check Status of Running Nodes in Hadoop

To see if nodes in your Hadoop cluster connected and run, followings are the ways to see

JPS Command

Run jps command in master node and slave nodes. In master node, the output of the jps should be something like this
$ jps
1634 JobHistoryServer
692 NameNode
2471 Jps
1191 ResourceManager
841 DataNode
1023 SecondaryNameNode
1311 NodeManager
When you run jps in slave, the output should be something like this
$ jps
528 Jps
292 NodeManager
170 DataNode

hdfs dfsadmin -report

This is a very nice tool to see the statistic of each nodes in your Hadoop cluster. When you run the command, you will get ouput something like
$ hdfs dfsadmin -report
Configured Capacity: 82999410688 (77.30 GB)
Present Capacity: 59326111744 (55.25 GB)
DFS Remaining: 59325284352 (55.25 GB)
DFS Used: 827392 (808 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
Pending deletion blocks: 0

-------------------------------------------------
Live datanodes (4):

Name: 10.0.0.251:50010 (localhost)
Hostname: localhost
Decommission Status : Normal
Configured Capacity: 20749852672 (19.32 GB)
DFS Used: 376832 (368 KB)
Non DFS Used: 5901377536 (5.50 GB)
DFS Remaining: 14831321088 (13.81 GB)
DFS Used%: 0.00%
DFS Remaining%: 71.48%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Apr 26 16:24:27 UTC 2018
Last Block Report: Thu Apr 26 16:06:36 UTC 2018


Name: 10.0.0.252:50010 (d2)
Hostname: d2
Decommission Status : Normal
Configured Capacity: 20749852672 (19.32 GB)
DFS Used: 368640 (360 KB)
Non DFS Used: 5901385728 (5.50 GB)
DFS Remaining: 14831321088 (13.81 GB)
DFS Used%: 0.00%
DFS Remaining%: 71.48%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Apr 26 16:24:27 UTC 2018
Last Block Report: Thu Apr 26 16:06:36 UTC 2018


Name: 10.0.0.253:50010 (d3)
Hostname: d3
Decommission Status : Normal
Configured Capacity: 20749852672 (19.32 GB)
DFS Used: 36864 (36 KB)
Non DFS Used: 5901717504 (5.50 GB)
DFS Remaining: 14831321088 (13.81 GB)
DFS Used%: 0.00%
DFS Remaining%: 71.48%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Apr 26 16:24:27 UTC 2018
Last Block Report: Thu Apr 26 16:06:36 UTC 2018


Name: 10.0.0.254:50010 (d4)
Hostname: d4
Decommission Status : Normal
Configured Capacity: 20749852672 (19.32 GB)
DFS Used: 45056 (44 KB)
Non DFS Used: 5901709312 (5.50 GB)
DFS Remaining: 14831321088 (13.81 GB)
DFS Used%: 0.00%
DFS Remaining%: 71.48%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Apr 26 16:24:27 UTC 2018
Last Block Report: Thu Apr 26 16:06:36 UTC 2018

Use web interface

If you like to work in black screen like me, this will be the last option for you. However, some of the people love this way... :D
To open the web interface, by default, it is in http://:8088
So, for example your server is 192.168.10.192, you can open the web interface using
http://192.168.10.192:8088

Comments

Popular posts from this blog

Find JIRA issues mentioned in Confluence Page

I have been walking through a lot of pages in internet but have not found any answer except one. However, the answer is not complete, so I will share my experience here. This feature is very useful, especially to summarize the issues found during certain tests, where the tests are reported in a confluence page. I found that there are so many questions about this, but Atlassian seems does not want to bother with this request. I found one way to do this by the following tricks Take one JIRA issue that related to the target confluence page (in this case, say it is GET-895) Find the global ID of a JIRA issue: http://bach.dc1.scram.com:8080/rest/api/latest/issue/GET-895/remotelink It will show the JSON like this: [{"id":28293,"self":"http://bach.dc1.scram.com:8080/rest/api/latest/issue/GET-895/remotelink/28293","globalId":"appId=662e1ccf-94da-3121-96ae-053d90587b29&pageId=105485659","application":{

Mininet/Containernet Problem: Exception: Error creating interface pair (s2-eth5,s3-eth1): RTNETLINK answers: File exists

If you did not shut down the previous running mininet/containernet network (e.g. if you lose your connection to remote server), you will got the following error when you try to rerun your mininet network Traceback (most recent call last): File "./mynet.py", line 31, in <module> net.addLink(d2, s1) File "build/bdist.linux-x86_64/egg/mininet/net.py", line 403, in addLink File "build/bdist.linux-x86_64/egg/mininet/link.py", line 430, in __init__ File "build/bdist.linux-x86_64/egg/mininet/link.py", line 474, in makeIntfPair File "build/bdist.linux-x86_64/egg/mininet/util.py", line 202, in makeIntfPair Exception: Error creating interface pair (d2-eth0,s1-eth2): RTNETLINK answers: File exists In order to solve the problem, you need to clean up the previous running topology by using the following command sudo mn -c It will clean up all your cache. It will be something like this $ sudo mn -c *** Re

Akte, paspor, dan visa untuk anak Indonesia yang lahir di Malaysia

Alhamdulillah, akhirnya urusan administrasi kenegaraan-nya Marwa sudah hampir selesai. Kemarin sore, di paspor marwa sudah ada visa pelancong selama 30 hari, tinggal nunggu proses di Putrajaya selama 7 hari untuk mendapatkan Multiple Entry-nya. Selepas itu, selesai.... Di dalam tulisan ini, saya pengen sedikit cerita pengalaman saya mengurus administrasi untuk anak Indonesia yang lahir di Malaysia (saya tidak tahu apakah untuk di negara lain prosedurnya juga sama). #1. Administrasi di klinik terdekat (sebelum melahirkan) Sebelum istri melahirkan, istri disarankan untuk mendaftar dan memeriksakan kandungannya secara rutin di klinik terdekat (meskipun hanya klinik, tapi fasilitas pemeriksaan dan labnya cukup lengkap dan canggih). Pada fase ini, istri akan diberi buku pemeriksaan. Buku ini penting untuk mengetahui kondisi si ibu, dan perkembangan janin. Buku ini akan diisi oleh dokter atau perawat yang memeriksa kandungan. #2. Administrasi setelah melahirkan Di klinik biasanya tid