{"id":6104,"date":"2024-08-19T09:01:01","date_gmt":"2024-08-19T01:01:01","guid":{"rendered":""},"modified":"2024-08-19T09:01:01","modified_gmt":"2024-08-19T01:01:01","slug":"\u591a\u53f0\u673a\u5668\u6267\u884c\u540c\u4e00\u547d\u4ee4_hadoop\u5b8c\u5168\u5206\u5e03\u5f0f\u642d\u5efa\u6b65\u9aa4","status":"publish","type":"post","link":"https:\/\/mushiming.com\/6104.html","title":{"rendered":"\u591a\u53f0\u673a\u5668\u6267\u884c\u540c\u4e00\u547d\u4ee4_hadoop\u5b8c\u5168\u5206\u5e03\u5f0f\u642d\u5efa\u6b65\u9aa4"},"content":{"rendered":"
\u3010\u7f16\u8005\u7684\u8bdd\u3011Docker\u6700\u6838\u5fc3\u7684\u7279\u6027\u4e4b\u4e00\uff0c\u5c31\u662f\u80fd\u591f\u5c06\u4efb\u4f55\u5e94\u7528\u5305\u62ecHadoop\u6253\u5305\u5230Docker\u955c\u50cf\u4e2d\u3002\u8fd9\u7bc7\u6559\u7a0b\u4ecb\u7ecd\u4e86\u5229\u7528Docker\u5728\u5355\u673a\u4e0a\u5feb\u901f\u642d\u5efa\u591a\u8282\u70b9Hadoop\u96c6\u7fa4\u7684\u8be6\u7ec6\u6b65\u9aa4\u3002\u4f5c\u8005\u5728\u53d1\u73b0\u76ee\u524d\u7684Hadoop on Docker\u9879\u76ee\u6240\u5b58\u5728\u7684\u95ee\u9898\u4e4b\u540e\uff0c\u5f00\u53d1\u4e86\u63a5\u8fd1\u6700\u5c0f\u5316\u7684Hadoop\u955c\u50cf\uff0c\u5e76\u4e14\u652f\u6301\u5feb\u901f\u642d\u5efa\u4efb\u610f\u8282\u70b9\u6570\u7684Hadoop\u96c6\u7fa4\u3002<\/span> \u8bf7\u67e5\u770b<\/span> \u57fa\u4e8eDocker\u642d\u5efaHadoop\u96c6\u7fa4\u4e4b\u5347\u7ea7\u7248 GitHub: <\/span> kiwanlau\/hadoop-cluster-docker \u672c\u9879\u76ee\u4e00\u5171\u5f00\u53d1\u4e864\u4e2a\u955c\u50cf\uff1a<\/span> 1. \u62c9\u53d6\u955c\u50cf<\/span> 1. \u51c6\u5907\u5de5\u4f5c<\/span>
\n
\n
<\/p>\n \u9879\u76ee\u5df2\u7ecf\u66f4\u65b0!!!<\/h3>\n
\n
\n
<\/p>\n \u4e00. \u9879\u76ee\u7b80\u4ecb<\/h5>\n
\n
\n
\n\u76f4\u63a5\u7528\u673a\u5668\u642d\u5efaHadoop\u96c6\u7fa4\u662f\u4e00\u4e2a\u76f8\u5f53\u75db\u82e6\u7684\u8fc7\u7a0b\uff0c\u5c24\u5176\u5bf9\u521d\u5b66\u8005\u6765\u8bf4\u3002\u4ed6\u4eec\u8fd8\u6ca1\u5f00\u59cb\u8dd1wordcount\uff0c\u53ef\u80fd\u5c31\u88ab\u8fd9\u4e2a\u95ee\u9898\u6298\u817e\u7684\u4f53\u65e0\u5b8c\u80a4\u4e86\u3002\u800c\u4e14\u4e5f\u4e0d\u662f\u6bcf\u4e2a\u4eba\u90fd\u6709\u597d\u51e0\u53f0\u673a\u5668\u5bf9\u5427\u3002\u4f60\u53ef\u4ee5\u5c1d\u8bd5\u7528\u591a\u4e2a\u865a\u62df\u673a\u642d\u5efa\uff0c\u524d\u63d0\u662f\u4f60\u6709\u4e2a\u6027\u80fd\u6760\u6760\u7684\u673a\u5668\u3002<\/span>
\n
\n
\n\u6211\u7684\u76ee\u6807\u662f\u5c06Hadoop\u96c6\u7fa4\u8fd0\u884c\u5728Docker\u5bb9\u5668\u4e2d\uff0c\u4f7fHadoop\u5f00\u53d1\u8005\u80fd\u591f\u5feb\u901f\u4fbf\u6377\u5730\u5728\u672c\u673a\u642d\u5efa\u591a\u8282\u70b9\u7684Hadoop\u96c6\u7fa4\u3002\u5176\u5b9e\u8fd9\u4e2a\u60f3\u6cd5\u5df2\u7ecf\u6709\u4e86\u4e0d\u5c11\u5b9e\u73b0\uff0c\u4f46\u662f\u90fd\u4e0d\u662f\u5f88\u7406\u60f3\uff0c\u4ed6\u4eec\u6216\u8005\u955c\u50cf\u592a\u5927\uff0c\u6216\u8005\u4f7f\u7528\u592a\u6162\uff0c\u6216\u8005\u4f7f\u7528\u4e86\u7b2c\u4e09\u65b9\u5de5\u5177\u4f7f\u5f97\u4f7f\u7528\u8d77\u6765\u8fc7\u4e8e\u590d\u6742\u3002\u4e0b\u8868\u4e3a\u4e00\u4e9b\u5df2\u77e5\u7684Hadoop on Docker\u9879\u76ee\u4ee5\u53ca\u5176\u5b58\u5728\u7684\u95ee\u9898\u3002<\/span>
\n
\n
<\/p>\n\u9879\u76ee \u955c\u50cf\u5927\u5c0f \u95ee\u9898 sequenceiq\/hadoop-docker:latest 1.491GB \u955c\u50cf\u592a\u5927\uff0c\u53ea\u652f\u6301\u5355\u4e2a\u8282\u70b9 sequenceiq\/hadoop-docker:2.7.0 1.76 GB sequenceiq\/hadoop-docker:2.60 1.624GB sequenceiq\/ambari:latest 1.782GB \u955c\u50cf\u592a\u5927\uff0c\u4f7f\u7528\u592a\u6162\uff0c\u4f7f\u7528\u590d\u6742 sequenceiq\/ambari:2.0.0 4.804GB sequenceiq\/ambari:latest:1.70 4.761GB alvinhenrick\/hadoop-mutinode 4.331GB \u955c\u50cf\u592a\u5927\uff0c\u6784\u5efa\u592a\u6162\uff0c\u589e\u52a0\u8282\u70b9\u9ebb\u70e6\uff0c\u6709bug <\/pre>\n
\n\u6211\u7684\u9879\u76ee\u53c2\u8003\u4e86alvinhenrick\/hadoop-mutinode\u9879\u76ee\uff0c\u4e0d\u8fc7\u6211\u505a\u4e86\u5927\u91cf\u7684\u4f18\u5316\u548c\u91cd\u6784\u3002alvinhenrick\/hadoop-mutinode\u9879\u76ee\u7684GitHub\u4e3b\u9875\u4ee5\u53ca\u4f5c\u8005\u6240\u5199\u7684\u535a\u5ba2\u5730\u5740\u5982\u4e0b\uff1a<\/span>
\n
<\/p>\n\n
\n\u4e0b\u9762\u4e24\u4e2a\u8868\u662falvinhenrick\/hadoop-mutinode\u9879\u76ee\u4e0e\u6211\u7684kiwenlau\/hadoop-cluster-docker\u9879\u76ee\u7684\u53c2\u6570\u5bf9\u6bd4\uff1a<\/span>
\n
\n
<\/p>\n\u955c\u50cf\u540d\u79f0 \u6784\u5efa\u65f6\u95f4 \u955c\u50cf\u5c42\u6570 \u955c\u50cf\u5927\u5c0f alvinhenrick\/serf 258.213s 21 239.4MB alvinhenrick\/hadoop-base 2236.055s 58 4.328GB alvinhenrick\/hadoop-dn 51.959s 74 4.331GB alvinhenrick\/hadoop-nn-dn 49.548s 84 4.331GB <\/pre>\n
<\/p>\n\u955c\u50cf\u540d\u79f0 \u6784\u5efa\u65f6\u95f4 \u955c\u50cf\u5c42\u6570 \u955c\u50cf\u5927\u5c0f kiwenlau\/serf-dnsmasq 509.46s 8 206.6 MB kiwenlau\/hadoop-base 400.29s 7 775.4 MB kiwenlau\/hadoop-master 5.41s 9 775.4 MB kiwenlau\/hadoop-slave 2.41s 8 775.4 MB <\/pre>\n
\n\u53ef\u77e5\uff0c\u6211\u4e3b\u8981\u4f18\u5316\u4e86\u8fd9\u6837\u51e0\u70b9\uff1a<\/span>
\n
<\/p>\n\n
\n\u66f4\u5feb\u66f4\u65b9\u4fbf\u5730\u6539\u53d8Hadoop\u96c6\u7fa4\u8282\u70b9\u6570\u76ee<\/span>
\n
\n\u53e6\u5916\uff0calvinhenrick\/hadoop-mutinode\u9879\u76ee\u589e\u52a0\u8282\u70b9\u65f6\u9700\u8981\u624b\u52a8\u4fee\u6539Hadoop\u914d\u7f6e\u6587\u4ef6\u7136\u540e\u91cd\u65b0\u6784\u5efahadoop-nn-dn\u955c\u50cf,\u7136\u540e\u4fee\u6539\u5bb9\u5668\u542f\u52a8\u811a\u672c\uff0c\u624d\u80fd\u5b9e\u73b0\u589e\u52a0\u8282\u70b9\u7684\u529f\u80fd\u3002\u800c\u6211\u901a\u8fc7shell\u811a\u672c\u5b9e\u73b0\u81ea\u52a8\u8bdd\uff0c\u4e0d\u52301\u5206\u949f\u53ef\u4ee5\u91cd\u65b0\u6784\u5efahadoop-master\u955c\u50cf\uff0c\u7136\u540e\u7acb\u5373\u8fd0\u884c\uff01\u672c\u9879\u76ee\u9ed8\u8ba4\u542f\u52a83\u4e2a\u8282\u70b9\u7684Hadoop\u96c6\u7fa4\uff0c\u652f\u6301\u4efb\u610f\u8282\u70b9\u6570\u7684Hadoop\u96c6\u7fa4\u3002<\/span>
\n
\n
\n\u53e6\u5916\uff0c\u542f\u52a8Hadoop\uff0c\u8fd0\u884cwordcount\u4ee5\u53ca\u91cd\u65b0\u6784\u5efa\u955c\u50cf\u90fd\u91c7\u7528\u4e86shell\u811a\u672c\u5b9e\u73b0\u81ea\u52a8\u5316\u3002\u8fd9\u6837\u4f7f\u5f97\u6574\u4e2a\u9879\u76ee\u7684\u4f7f\u7528\u4ee5\u53ca\u5f00\u53d1\u90fd\u53d8\u5f97\u975e\u5e38\u65b9\u4fbf\u5feb\u6377\u3002<\/span>
\n
\n
\n\u5f00\u53d1\u6d4b\u8bd5\u73af\u5883<\/span>
\n
<\/p>\n\n
\n\u5c0f\u4f19\u4f34\u4eec\uff0c\u786c\u76d8\u4e0d\u591f\uff0c\u5185\u5b58\u4e0d\u591f\uff0c\u5c24\u5176\u662f\u5185\u6838\u7248\u672c\u8fc7\u4f4e\u4f1a\u5bfc\u81f4\u8fd0\u884c\u5931\u8d25\u3002<\/span>
\n
\n
<\/p>\n \u4e8c. \u955c\u50cf\u7b80\u4ecb<\/h5>\n
\n
<\/p>\n\n
\nserf-dnsmasq\u955c\u50cf<\/span>
\n
<\/p>\n\n
\n\u5bb9\u5668\u542f\u52a8\u65f6\uff0cmaster\u8282\u70b9\u7684IP\u4f1a\u4f20\u7ed9\u6240\u6709slave\u8282\u70b9\u3002serf\u4f1a\u5728container\u542f\u52a8\u540e\u7acb\u5373\u542f\u52a8\u3002slave\u8282\u70b9\u4e0a\u7684serf agent\u4f1a\u9a6c\u4e0a\u53d1\u73b0master\u8282\u70b9\uff08master IP\u5b83\u4eec\u90fd\u77e5\u9053\u561b\uff09\uff0cmaster\u8282\u70b9\u5c31\u9a6c\u4e0a\u53d1\u73b0\u4e86\u6240\u6709slave\u8282\u70b9\u3002\u7136\u540e\u5b83\u4eec\u4e4b\u95f4\u901a\u8fc7\u4e92\u76f8\u4ea4\u6362\u4fe1\u606f\uff0c\u6240\u6709\u8282\u70b9\u5c31\u80fd\u77e5\u9053\u5176\u4ed6\u6240\u6709\u8282\u70b9\u7684\u5b58\u5728\u4e86\u3002\uff08Everyone will know Everyone\uff09\u3002serf\u53d1\u73b0\u65b0\u7684\u8282\u70b9\u65f6\uff0c\u5c31\u4f1a\u91cd\u65b0\u914d\u7f6ednsmasq\uff0c\u7136\u540e\u91cd\u542fdnsmasq\u3002\u6240\u4ee5dnsmasq\u5c31\u80fd\u591f\u89e3\u6790\u96c6\u7fa4\u7684\u6240\u6709\u8282\u70b9\u7684\u57df\u540d\u5566\u3002\u8fd9\u4e2a\u8fc7\u7a0b\u968f\u7740\u8282\u70b9\u7684\u589e\u52a0\u4f1a\u8017\u65f6\u66f4\u4e45\uff0c\u56e0\u6b64\uff0c\u82e5\u914d\u7f6e\u7684Hadoop\u8282\u70b9\u6bd4\u8f83\u591a\uff0c\u5219\u5728\u542f\u52a8\u5bb9\u5668\u540e\u9700\u8981\u6d4b\u8bd5serf\u662f\u5426\u53d1\u73b0\u4e86\u6240\u6709\u8282\u70b9\uff0cDNS\u662f\u5426\u80fd\u591f\u89e3\u6790\u6240\u6709\u8282\u70b9\u57df\u540d\u3002\u7a0d\u7b49\u7247\u523b\u624d\u80fd\u542f\u52a8Hadoop\u3002\u8fd9\u4e2a\u89e3\u51b3\u65b9\u6848\u662f\u7531SequenceIQ\u516c\u53f8\u63d0\u51fa\u7684\uff0c\u8be5\u516c\u53f8\u4e13\u6ce8\u4e8e\u5c06Hadoop\u8fd0\u884c\u5728Docker\u4e2d\u3002\u53c2\u8003\u8fd9\u4e2a<\/span> \u6f14\u8bb2\u7a3f
\n\u3002<\/span>
\n
\n
\nhadoop-base\u955c\u50cf<\/span>
\n <\/span>
\n
<\/p>\n\n
\n\u53e6\u5916\uff0c\u7f16\u8bd1Hadoop\u7684\u6b65\u9aa4\u8bf7\u53c2\u8003\u6211\u7684<\/span> \u535a\u5ba2
\n\u3002<\/span>
\n
\n
\n\u5982\u679c\u9700\u8981\u91cd\u65b0\u5f00\u53d1\u6211\u7684hadoop-base, \u9700\u8981\u4e0b\u8f7d\u7f16\u8bd1\u8fc7\u7684hadoop-2.3.0\u5b89\u88c5\u5305\uff0c\u653e\u5230hadoop-cluster-docker\/hadoop-base\/files\u76ee\u5f55\u5185\u3002\u6211\u7f16\u8bd1\u768464\u4f4dhadoop-2.3.0\u4e0b\u8f7d\u5730\u5740\uff1a<\/span> hadoop-2.3.0
\n
\n
\n\u53e6\u5916\uff0c\u6211\u8fd8\u7f16\u8bd1\u4e8664\u4f4d\u7684Hadoop 2.5.2\u30012.6.0,\u30012.7.0, \u5176\u4e0b\u8f7d\u5730\u5740\u5982\u4e0b\uff1a<\/span>
\n
\n- <\/span> hadoop-2.3.0
\n
\n- <\/span> hadoop-2.5.2
\n
\n- <\/span> hadoop-2.6.0
\n
\n- <\/span> hadoop-2.7.0
\n
\n
\nhadoop-master\u955c\u50cf<\/span>
\n
<\/p>\n\n
\n\u8fd9\u4e00\u6b65\u9700\u8981\u914d\u7f6eslaves\u6587\u4ef6\uff0c\u800cslaves\u6587\u4ef6\u9700\u8981\u5217\u51fa\u6240\u6709\u8282\u70b9\u7684\u57df\u540d\u6216\u8005IP\u3002\u56e0\u6b64\uff0cHadoop\u8282\u70b9\u6570\u76ee\u4e0d\u540c\u65f6\uff0cslaves\u6587\u4ef6\u81ea\u7136\u4e5f\u4e0d\u4e00\u6837\u3002\u56e0\u6b64\uff0c\u66f4\u6539Hadoop\u96c6\u7fa4\u8282\u70b9\u6570\u76ee\u65f6\uff0c\u9700\u8981\u4fee\u6539slaves\u6587\u4ef6\u7136\u540e\u91cd\u65b0\u6784\u5efahadoop-master\u955c\u50cf\u3002\u6211\u7f16\u5199\u4e86\u4e00\u4e2aresize-cluster.sh\u811a\u672c\u81ea\u52a8\u5316\u8fd9\u4e00\u8fc7\u7a0b\u3002\u4ec5\u9700\u7ed9\u5b9a\u8282\u70b9\u6570\u76ee\u4f5c\u4e3a\u811a\u672c\u53c2\u6570\u5c31\u53ef\u4ee5\u8f7b\u677e\u5b9e\u73b0Hadoop\u96c6\u7fa4\u8282\u70b9\u6570\u76ee\u7684\u66f4\u6539\u3002\u7531\u4e8ehadoop-master\u955c\u50cf\u4ec5\u4ec5\u505a\u4e00\u4e9b\u914d\u7f6e\u5de5\u4f5c\uff0c\u4e5f\u65e0\u9700\u4e0b\u8f7d\u4efb\u4f55\u6587\u4ef6\uff0c\u6574\u4e2a\u8fc7\u7a0b\u975e\u5e38\u5feb\uff0c1\u5206\u949f\u5c31\u8db3\u591f\u4e86\u3002<\/span>
\n
\n
\nhadoop-slave\u955c\u50cf<\/span>
\n
<\/p>\n\n
\n\u955c\u50cf\u5927\u5c0f\u5206\u6790<\/span>
\n
\n
\n\u4e0b\u8868\u4e3a<\/span>
\nsudo docker images<\/code>
\n\u7684\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE index.alauda.cn\/kiwenlau\/hadoop-slave 0.1.0 dc03 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/hadoop-master 0.1.0 7c9d32ede450 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/hadoop-base 0.1.0 5571bd5de58e 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/serf-dnsmasq 0.1.0 09ed89c24ee8 17 hours ago 206.7 MB ubuntu 15.04 bd94ae 3 weeks ago 131.3 MB <\/pre>\n
\n\u6613\u77e5\u4ee5\u4e0b\u51e0\u4e2a\u7ed3\u8bba\uff1a<\/span>
\n
<\/p>\n\n
\n\u4e0b\u8868\u4e3a<\/span>
\nsudo docker history index.alauda.cn\/kiwenlau\/hadoop-base:0.1.0<\/code>
\n\u7684\u90e8\u5206\u8fd0\u884c\u7ed3\u679c<\/span>
\n
<\/p>\nIMAGE CREATED CREATED BY SIZE 2039b9b81146 44 hours ago \/bin\/sh -c #(nop) ADD multi:a93c971a49514e787 158.5 MB cdbf30 44 hours ago \/bin\/sh -c apt-get install -y openjdk-7-jdk 324.6 MB da7d10c790c1 44 hours ago \/bin\/sh -c apt-get install -y openssh-server 87.58 MB c65cb568defc 44 hours ago \/bin\/sh -c curl -Lso serf.zip https:\/\/dl.bint 14.46 MB 3e22b3d72e33 44 hours ago \/bin\/sh -c apt-get update && apt-get install 60.89 MB b68f8c8d2140 3 weeks ago \/bin\/sh -c #(nop) ADD file:d90f7467c470bfa9a3 131.3 MB <\/pre>\n
\n\u53ef\u77e5\uff1a<\/span>
\n
<\/p>\n\n
\n\u56e0\u6b64\uff0c\u6211\u6240\u5f00\u53d1\u7684hadoop\u955c\u50cf\u4ee5\u53ca\u63a5\u8fd1\u6700\u5c0f\uff0c\u4f18\u5316\u7a7a\u95f4\u5df2\u7ecf\u5f88\u5c0f\u4e86\u3002<\/span>
\n
\n
\n\u4e0b\u56fe\u663e\u793a\u4e86\u8fd9\u4e2a\u9879\u76ee\u7684\u955c\u50cf\u7ed3\u6784\uff1a<\/span>
\n
<\/p>\n
\n
<\/p>\n \u4e09. 3\u8282\u70b9Hadoop\u96c6\u7fa4\u642d\u5efa\u6b65\u9aa4<\/h5>\n
\n
<\/p>\nsudo docker pull index.alauda.cn\/kiwenlau\/hadoop-master:0.1.0 sudo docker pull index.alauda.cn\/kiwenlau\/hadoop-slave:0.1.0 sudo docker pull index.alauda.cn\/kiwenlau\/hadoop-base:0.1.0 sudo docker pull index.alauda.cn\/kiwenlau\/serf-dnsmasq:0.1.0 <\/pre>\n
\n
\n3~5\u5206\u949fOK~\u4e5f\u53ef\u4ee5\u76f4\u63a5\u4ece\u6211\u7684DokcerHub\u4ed3\u5e93\u4e2d\u62c9\u53d6\u955c\u50cf\uff0c\u8fd9\u6837\u5c31\u53ef\u4ee5\u8df3\u8fc7\u7b2c2\u6b65\uff1a<\/span>
\n
<\/p>\nsudo docker pull kiwenlau\/hadoop-master:0.1.0 sudo docker pull kiwenlau\/hadoop-slave:0.1.0 sudo docker pull kiwenlau\/hadoop-base:0.1.0 sudo docker pull kiwenlau\/serf-dnsmasq:0.1.0 <\/pre>\n
\n
\n\u67e5\u770b\u4e0b\u8f7d\u7684\u955c\u50cf\uff1a<\/span>
\n
<\/p>\nsudo docker images <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE index.alauda.cn\/kiwenlau\/hadoop-slave 0.1.0 dc03 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/hadoop-master 0.1.0 7c9d32ede450 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/hadoop-base 0.1.0 5571bd5de58e 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/serf-dnsmasq 0.1.0 09ed89c24ee8 17 hours ago 206.7 MB <\/pre>\n
\n
\n\u5176\u4e2dhadoop-base\u955c\u50cf\u662f\u57fa\u4e8eserf-dnsmasq\u955c\u50cf\u7684\uff0chadoop-slave\u955c\u50cf\u548chadoop-master\u955c\u50cf\u90fd\u662f\u57fa\u4e8ehadoop-base\u955c\u50cf\u3002\u6240\u4ee5\u5176\u5b9e4\u4e2a\u955c\u50cf\u4e00\u5171\u4e5f\u5c31777.4MB\u3002<\/span>
\n
\n
\n2. \u4fee\u6539\u955c\u50cftag<\/span>
\n
<\/p>\nsudo docker tag dc03 kiwenlau\/hadoop-slave:0.1.0 sudo docker tag 7c9d32ede450 kiwenlau\/hadoop-master:0.1.0 sudo docker tag 5571bd5de58e kiwenlau\/hadoop-base:0.1.0 sudo docker tag 09ed89c24ee8 kiwenlau\/serf-dnsmasq:0.1.0 <\/pre>\n
\n\u67e5\u770b\u4fee\u6539tag\u540e\u955c\u50cf\uff1a<\/span>
\n
<\/p>\nsudo docker images <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nREPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE index.alauda.cn\/kiwenlau\/hadoop-slave 0.1.0 dc03 17 hours ago 777.4 MB kiwenlau\/hadoop-slave 0.1.0 dc03 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/hadoop-master 0.1.0 7c9d32ede450 17 hours ago 777.4 MB kiwenlau\/hadoop-master 0.1.0 7c9d32ede450 17 hours ago 777.4 MB kiwenlau\/hadoop-base 0.1.0 5571bd5de58e 17 hours ago 777.4 MB index.alauda.cn\/kiwenlau\/hadoop-base 0.1.0 5571bd5de58e 17 hours ago 777.4 MB kiwenlau\/serf-dnsmasq 0.1.0 09ed89c24ee8 17 hours ago 206.7 MB index.alauda.cn\/kiwenlau\/serf-dnsmasq 0.1.0 09ed89c24ee8 17 hours ago 206.7 MB <\/pre>\n
\n\u4e4b\u6240\u4ee5\u8981\u4fee\u6539\u955c\u50cf\uff0c\u662f\u56e0\u4e3a\u6211\u9ed8\u8ba4\u662f\u5c06\u955c\u50cf\u4e0a\u4f20\u5230Dockerhub, \u56e0\u6b64Dokerfile\u4ee5\u53cashell\u811a\u672c\u4e2d\u5f97\u955c\u50cf\u540d\u79f0\u90fd\u662f\u6ca1\u6709alauada\u524d\u7f00\u7684\uff0csorry for this....\u4e0d\u8fc7\u6539tag\u8fd8\u662f\u5f88\u5feb\u6ef4\u3002\u82e5\u76f4\u63a5\u4e0b\u8f7d\u6211\u5728DockerHub\u4e2d\u7684\u955c\u50cf\uff0c\u81ea\u7136\u5c31\u4e0d\u9700\u8981\u4fee\u6539tag...\u4e0d\u8fc7Alauda\u955c\u50cf\u4e0b\u8f7d\u901f\u5ea6\u5f88\u5feb\u7684\u54c8~<\/span>
\n
\n
\n3.\u4e0b\u8f7d\u6e90\u4ee3\u7801<\/span>
\n
\n
<\/p>\ngit clone https:\/\/github.com\/kiwenlau\/hadoop-cluster-docker <\/pre>\n
\n\u4e3a\u4e86\u9632\u6b62GitHub\u88abXX\uff0c\u6211\u628a\u4ee3\u7801\u5bfc\u5165\u5230\u4e86\u5f00\u6e90\u4e2d\u56fd\u7684Git\u4ed3\u5e93\uff1a<\/span>
\n
<\/p>\ngit clone http:\/\/git.oschina.net\/kiwenlau\/hadoop-cluster-docker <\/pre>\n
\n4. \u8fd0\u884c\u5bb9\u5668<\/span>
\n
<\/p>\ncd hadoop-cluster-docker .\/start-container.sh <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nstart master container... start slave1 container... start slave2 container... root@master:~# <\/pre>\n
\n\u4e00\u5171\u5f00\u542f\u4e863\u4e2a\u5bb9\u5668\uff0c1\u4e2amaster, 2\u4e2aslave\u3002\u5f00\u542f\u5bb9\u5668\u540e\u5c31\u8fdb\u5165\u4e86master\u5bb9\u5668root\u7528\u6237\u7684\u6839\u76ee\u5f55\uff08\/root\uff09\u3002<\/span>
\n
\n
\n\u67e5\u770bmaster\u7684root\u7528\u6237\u5bb6\u76ee\u5f55\u7684\u6587\u4ef6\uff1a<\/span>
\n
<\/p>\nls <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nhdfs run-wordcount.sh serf_log start-hadoop.sh start-ssh-serf.sh <\/pre>\n
\nstart-hadoop.sh\u662f\u5f00\u542fhadoop\u7684shell\u811a\u672c\uff0crun-wordcount.sh\u662f\u8fd0\u884cwordcount\u7684shell\u811a\u672c\uff0c\u53ef\u4ee5\u6d4b\u8bd5\u955c\u50cf\u662f\u5426\u6b63\u5e38\u5de5\u4f5c\u3002<\/span>
\n
\n
\n5.\u6d4b\u8bd5\u5bb9\u5668\u662f\u5426\u6b63\u5e38\u542f\u52a8\uff08\u6b64\u65f6\u5df2\u8fdb\u5165master\u5bb9\u5668\uff09<\/span>
\n
\n
\n\u67e5\u770bhadoop\u96c6\u7fa4\u6210\u5458\uff1a<\/span>
\n
<\/p>\nserf members <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nmaster.kiwenlau.com 172.17.0.65:7946 alive slave1.kiwenlau.com 172.17.0.66:7946 alive slave2.kiwenlau.com 172.17.0.67:7946 alive <\/pre>\n
\n\u82e5\u7ed3\u679c\u7f3a\u5c11\u8282\u70b9\uff0c\u53ef\u4ee5\u7a0d\u7b49\u7247\u523b\uff0c\u518d\u6267\u884c\u201cserf members\u201d\u547d\u4ee4\u3002\u56e0\u4e3aserf agent\u9700\u8981\u65f6\u95f4\u53d1\u73b0\u6240\u6709\u8282\u70b9\u3002<\/span>
\n
\n
\n\u6d4b\u8bd5ssh\uff1a<\/span>
\n
<\/p>\nssh slave2.kiwenlau.com <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nWarning: Permanently added 'slave2.kiwenlau.com,172.17.0.67' (ECDSA) to the list of known hosts. Welcome to Ubuntu 15.04 (GNU\/Linux 3.13.0-53-generic x86_64) * Documentation: https:\/\/help.ubuntu.com\/ The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in \/usr\/share\/doc\/*\/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. root@slave2:~# <\/pre>\n
\n\u9000\u51faslave2\uff1a<\/span>
\n
<\/p>\nexit <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\nlogout Connection to slave2.kiwenlau.com closed. <\/pre>\n
\n\u82e5ssh\u5931\u8d25\uff0c\u8bf7\u7a0d\u7b49\u7247\u523b\u518d\u6d4b\u8bd5\uff0c\u56e0\u4e3adnsmasq\u7684dns\u670d\u52a1\u5668\u542f\u52a8\u9700\u8981\u65f6\u95f4\u3002\u6d4b\u8bd5\u6210\u529f\u540e\uff0c\u5c31\u53ef\u4ee5\u5f00\u542fHadoop\u96c6\u7fa4\u4e86\uff01\u5176\u5b9e\u4f60\u4e5f\u53ef\u4ee5\u4e0d\u8fdb\u884c\u6d4b\u8bd5\uff0c\u5f00\u542f\u5bb9\u5668\u540e\u8010\u5fc3\u7b49\u5f85\u4e00\u5206\u949f\u5373\u53ef\uff01<\/span>
\n
\n
\n6. \u5f00\u542fHadoop<\/span>
\n
<\/p>\n.\/start-hadoop.sh <\/pre>\n
\n\u4e0a\u4e00\u6b65ssh\u5230slave2\u4e4b\u540e\uff0c\u8bf7\u8bb0\u5f97\u56de\u5230master\u554a\uff01\u8fd0\u884c\u7ed3\u679c\u592a\u591a\uff0c\u5ffd\u7565\uff0cHadoop\u7684\u542f\u52a8\u901f\u5ea6\u53d6\u51b3\u4e8e\u673a\u5668\u6027\u80fd....<\/span>
\n
\n
\n7. \u8fd0\u884cwordcount<\/span>
\n
\n
<\/p>\n.\/run-wordcount.sh <\/pre>\n
\n\u8fd0\u884c\u7ed3\u679c\uff1a<\/span>
\n
<\/p>\ninput file1.txt: Hello Hadoop input file2.txt: Hello Docker wordcount output: Docker 1 Hadoop 1 Hello 2 <\/pre>\n
\nwordcount\u7684\u6267\u884c\u901f\u5ea6\u53d6\u51b3\u4e8e\u673a\u5668\u6027\u80fd....<\/span>
\n
\n
<\/p>\n \u56db. N\u8282\u70b9Hadoop\u96c6\u7fa4\u642d\u5efa\u6b65\u9aa4<\/h5>\n
\n
<\/p>\n\n
\n2. \u91cd\u65b0\u6784\u5efahadoop-master\u955c\u50cf<\/span>
\n
\n
<\/p>\n.\/resize-cluster.sh 5 <\/pre>\n
<\/p>\n\n
\n3. \u542f\u52a8\u5bb9\u5668<\/span>
\n
\n
<\/p>\n.\/start-container.sh 5 <\/pre>\n
<\/p>\n\n
\n4. \u6d4b\u8bd5\u5de5\u4f5c<\/span>
\n
<\/p>\n\n
\n==================================<\/span>
\n
\n\u4f5c\u8005\u4ecb\u7ecd<\/span>
\n
\n\u5218\u51ef(KiwenLau)\uff1a\u6bd5\u4e1a\u4e8e\u4e2d\u56fd\u79d1\u5b66\u6280\u672f\u5927\u5b66\uff0c\u76ee\u524d\u5728\u65e5\u672c\u56fd\u7acb\u4fe1\u606f\u5b66\u7814\u7a76\u6240\u653b\u8bfb\u4e91\u8ba1\u7b97\u65b9\u5411\u7684\u535a\u58eb\u5b66\u4f4d\uff0c\u8fd1\u671f\u4e13\u6ce8\u4e8eDocker\u6280\u672f\u7684\u7814\u7a76\u3002\u4e2a\u4eba\u7ad9\u70b9\uff1a<\/span> GitHub
\n, <\/span> \u535a\u5ba2<\/p>\n","protected":false},"excerpt":{"rendered":"\u591a\u53f0\u673a\u5668\u6267\u884c\u540c\u4e00\u547d\u4ee4_hadoop\u5b8c\u5168\u5206\u5e03\u5f0f\u642d\u5efa\u6b65\u9aa4\u3010\u7f16\u8005\u7684\u8bdd\u3011Docker\u6700\u6838\u5fc3\u7684\u7279\u6027\u4e4b\u4e00\uff0c\u5c31\u662f\u80fd\u591f\u5c06\u4efb\u4f55\u5e94\u7528\u5305\u62ecHadoop\u6253\u5305\u5230Do...","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"_links":{"self":[{"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/posts\/6104"}],"collection":[{"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/comments?post=6104"}],"version-history":[{"count":0,"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/posts\/6104\/revisions"}],"wp:attachment":[{"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/media?parent=6104"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/categories?post=6104"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mushiming.com\/wp-json\/wp\/v2\/tags?post=6104"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}