Yarn命令使用及wordcount解析

前言：

前面几篇博客主要介绍了MapReduce与Yarn的架构设计及简单工作流程，本篇文章将以wordcount程序为例，简单介绍下Yarn的使用。

1.wordcount示例运行

[root@hadoop000 ~]# su - hadoop[hadoop@hadoop000 ~]$ jps9201 SecondaryNameNode9425 ResourceManager13875 Jps9540 NodeManager8852 NameNode8973 DataNode# 创建wordcount目录[hadoop@hadoop000 ~]$ hdfs dfs -mkdir -p /wordcount/input[hadoop@hadoop000 ~]$ vi test.logjepson ruozehero yimi xjp123a b a[hadoop@hadoop000 ~]$ hdfs dfs -put test.log /wordcount/input[hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/input           Found 1 items-rw-r--r--   1 hadoop supergroup         37 2018-05-29 20:38 /wordcount/input/test.log# 执行wordcount示例jar包[hadoop@hadoop000 ~]$ yarn jar \> /opt/software/hadoop-2.8.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar \> wordcount \> /wordcount/input \> /wordcount/output18/05/29 20:40:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:803218/05/29 20:40:59 INFO input.FileInputFormat: Total input files to process : 118/05/29 20:41:00 INFO mapreduce.JobSubmitter: number of splits:118/05/29 20:41:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526991305992_000118/05/29 20:41:01 INFO impl.YarnClientImpl: Submitted application application_1526991305992_000118/05/29 20:41:01 INFO mapreduce.Job: The url to track the job: http://hadoop000:8088/proxy/application_1526991305992_0001/18/05/29 20:41:01 INFO mapreduce.Job: Running job: job_1526991305992_000118/05/29 20:41:14 INFO mapreduce.Job: Job job_1526991305992_0001 running in uber mode : false18/05/29 20:41:14 INFO mapreduce.Job:  map 0% reduce 0/05/29 20:41:23 INFO mapreduce.Job:  map 100% reduce 0/05/29 20:41:29 INFO mapreduce.Job:  map 100% reduce 100/05/29 20:41:30 INFO mapreduce.Job: Job job_1526991305992_0001 completed successfully18/05/29 20:41:30 INFO mapreduce.Job: Counters: 49# 查看结果[hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/outputFound 2 items-rw-r--r--   1 hadoop supergroup          0 2018-05-29 20:41 /wordcount/output/_SUCCESS-rw-r--r--   1 hadoop supergroup         51 2018-05-29 20:41 /wordcount/output/part-r-00000[hadoop@hadoop000 ~]$ hdfs dfs -cat /wordcount/output/part-r-00000123     1a       2b       1hero    1jepson  1ruoze   1xjp     1yimi    1

登录网页查看相关信息：http://192.168.6.217:8088/cluster

2.Yarn常用命令总结

yarn jar               --run a jar fileyarn application -list      --列出在跑的jobyarn application -kill application_1526991305992_0001（job的id） --杀掉在跑的job

3.wordcount流程详解

参考：https://blog.csdn.net/yczws1/article/details/21794873