Differences

This shows you the differences between two versions of the page.

--- tutorials:cb1:customization:deploying_hadoop_cluster_on_cubieboard_guide [2013/10/17 11:44]
admin
+++ tutorials:cb1:customization:deploying_hadoop_cluster_on_cubieboard_guide [2013/12/23 14:50] (current)
@@ Line 86: / Line 86: @@
 Stay here to sure all cbs have an user with Static IP which can ping each other.
 ====SSH sever ====
-master <--no passwd-->slave1
-master <--no passwd-->slave2
+master<no passwd>slave1
-master <--no passwd-->slave3
-B--no passwd-->A
+master<no passwd>slave2
+master<no passwd>slave3
+B  ---no passwd-->A
 A
 <code>
@@ Line 106: / Line 111: @@
 rm –r ~/id_rsa.pub
 </code>
+master to slave1
 <code>
 hadoop@master:~$
@@ Line 113: / Line 119: @@
  * Documentation:  https://wiki.linaro.org/
 Last login: Thu Oct 17 03:38:36 2013 from master
-hadoop@slave1:~$
+</code>
+slave1 to master
+<code>
+hadoop@slave1:~$ ssh master
+Welcome to Linaro 13.04 (GNU/Linux 3.4.43+ armv7l)
+ * Documentation:  https://wiki.linaro.org/
+Last login: Thu Oct 17 03:38:58 2013 from slave1
 </code>
 ====JDK====
-===JDK path modification===
+==JDK path modification==
+''hadoop@master:~$ vim /etc/profile''
 '' #export JAVA_HOME=/lib/jdk''
+You also should add to other nodes.
 ====Hadoop configuration====
-===Hadoop configuration parameters===
+You should edit core-site.xml hdfs-site.xml mapred-site.xml on /hadoop/hadoop_0.20.203_master/conf for master
-You should edit core-site.xml hdfs-site.xml mapred-site.xml on /hadoop/hadoop_0.20.203_master/conf
+You can do Hadoop configuration on your host computer .
+<code>
+aaron@cubietech:/work$ sudo mv hadoop_0.20.203 hadoop
+aaron@cubietech:/work$ cd hadoop/conf/
+aaron@cubietech:/work/hadoop/conf$ sudo vim core-site.xml
+</code>
 core-site.xml
@@ Line 169: / Line 194: @@
 </configuration>
 </code>
-===7.How to run===
+After that ,You should copy hadoop to every node
 <code>
-bin/hadoop namenode -format
+scp -r hadoop root@192.168.1.40:/usr/local
+scp -r hadoop root@192.168.1.41:/usr/local
+scp -r hadoop root@192.168.1.42:/usr/local
+scp -r hadoop root@192.168.1.43:/usr/local
+</code>
+====How to run====
+''hadoop@master:~$ cd /usr/local/hadoop/''
+format filesys
+''bin/hadoop namenode -format ''
+<code>
+hadoop@master:/usr/local/hadoop$ bin/hadoop namenode -format
+/10/17 05:49:16 INFO namenode.NameNode: STARTUP_MSG:
+/************************************************************
+STARTUP_MSG: Starting NameNode
+STARTUP_MSG:   host = master/192.168.1.40
+STARTUP_MSG:   args = [-format]
+STARTUP_MSG:   version = 0.20.203.0
+STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333; compiled by 'oom' on Wed May  4 07:57:50 PDT 2011
+************************************************************/
+Re-format filesystem in /usr/local/hadoop/datalog1 ? (Y or N) Y
+Re-format filesystem in /usr/local/hadoop/datalog2 ? (Y or N) Y
+/10/17 05:49:22 INFO util.GSet: VM type       = 32-bit
+/10/17 05:49:22 INFO util.GSet: 2% max memory = 19.335 MB
+/10/17 05:49:22 INFO util.GSet: capacity      = 2^22 = 4194304 entries
+/10/17 05:49:22 INFO util.GSet: recommended=4194304, actual=4194304
+/10/17 05:49:24 INFO namenode.FSNamesystem: fsOwner=hadoop
+/10/17 05:49:24 INFO namenode.FSNamesystem: supergroup=supergroup
+/10/17 05:49:24 INFO namenode.FSNamesystem: isPermissionEnabled=true
+/10/17 05:49:24 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
+/10/17 05:49:24 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
+/10/17 05:49:24 INFO namenode.NameNode: Caching file names occuring more than 10 times
+/10/17 05:49:26 INFO common.Storage: Image file of size 112 saved in 0 seconds.
+/10/17 05:49:26 INFO common.Storage: Storage directory /usr/local/hadoop/datalog1 has been successfully formatted.
+/10/17 05:49:26 INFO common.Storage: Image file of size 112 saved in 0 seconds.
+/10/17 05:49:27 INFO common.Storage: Storage directory /usr/local/hadoop/datalog2 has been successfully formatted.
+/10/17 05:49:27 INFO namenode.NameNode: SHUTDOWN_MSG:
+/************************************************************
+SHUTDOWN_MSG: Shutting down NameNode at master/192.168.1.40
+************************************************************/
+</code>
+<code>
 bin/hadoop dfsadmin -report
@@ Line 179: / Line 247: @@
 bin/stop-all.sh // stop
-./bin/hadoop jar hadoop-examples-0.20.203.0.jar pi 100 100 //Calculates PI
 </code>
-====Troubleshooting====
-===== You can also read this tutorial =====
+''./bin/hadoop jar hadoop-examples-0.20.203.0.jar pi 100 100 ''
+{{:tutorials:cb1:installation:screenshot_from_2013-07-31_18_18_35.png|800}}
+You can also see filesys on web
+http:192.168.1.2:50030
+http:192.168.1.2:50070
+{{:tutorials:cb1:installation:screenshot_from_2013-07-31_18_18_14.png|800}}
+You can also read this tutorial
 http://www.cnblogs.com/xia520pi/archive/2012/05/16/2503949.html
 <WRAP noprint>
 {{tag>Cubieboard Cubietruck}}
 </WRAP>

CubieBoard Docs

User Tools

Site Tools

Differences

Page Tools