Where my life experiences are shared

I had difficulty in finding any large text file. In Google, some of them suggest to download from wikimedia dump and some of them from gutenberg project. However, the following steps are a lot easier than going to all the menus in either wikimedia or gutenberg. Enter any small text file In the first step, just enter any small text file. For example: $ echo 'arwan ahmad khoiruddin' > mytext.txt Merge the text into big file Next, you can merge the text into big file. Here's how $ cat mytext.txt mytext.txt mytext.txt > merged.txt The content of file merged.txt will be arwan ahmad khoiruddin arwan ahmad khoiruddin arwan ahmad khoiruddin So, if you want to create big file, you can just repeat the cat as many as possible. For example $ cat mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt mytext.txt ...

To see if nodes in your Hadoop cluster connected and run, followings are the ways to see JPS Command Run jps command in master node and slave nodes. In master node, the output of the jps should be something like this $ jps 1634 JobHistoryServer 692 NameNode 2471 Jps 1191 ResourceManager 841 DataNode 1023 SecondaryNameNode 1311 NodeManager When you run jps in slave, the output should be something like this $ jps 528 Jps 292 NodeManager 170 DataNode hdfs dfsadmin -report This is a very nice tool to see the statistic of each nodes in your Hadoop cluster. When you run the command, you will get ouput something like $ hdfs dfsadmin -report Configured Capacity: 82999410688 (77.30 GB) Present Capacity: 59326111744 (55.25 GB) DFS Remaining: 59325284352 (55.25 GB) DFS Used: 827392 (808 KB) DFS Used%: 0.00% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 Pending deletion blocks: 0 ...