install hadoop-2.5
?
?
logs from mapreduce:
hadoop@ubuntu:/usr/local/hadoop/hadoop-2.5.1$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar wordcount wc wc-out
14/11/05 18:19:23 INFO client.RMProxy: Connecting to ResourceManager at namenode/192.168.1.25:8032
14/11/05 18:19:24 INFO input.FileInputFormat: Total input paths to process : 22
14/11/05 18:19:24 INFO mapreduce.JobSubmitter: number of splits:22
14/11/05 18:19:24 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1415182439385_0001
14/11/05 18:19:25 INFO impl.YarnClientImpl: Submitted application application_1415182439385_0001
14/11/05 18:19:25 INFO mapreduce.Job: The url to track the job: http://namenode:50030/proxy/application_1415182439385_0001/
14/11/05 18:19:25 INFO mapreduce.Job: Running job: job_1415182439385_0001
14/11/05 18:19:32 INFO mapreduce.Job: Job job_1415182439385_0001 running in uber mode : false
14/11/05 18:19:32 INFO mapreduce.Job: ?map 0% reduce 0%
14/11/05 18:19:44 INFO mapreduce.Job: ?map 9% reduce 0%
14/11/05 18:19:45 INFO mapreduce.Job: ?map 27% reduce 0%
14/11/05 18:19:54 INFO mapreduce.Job: ?map 32% reduce 0%
14/11/05 18:19:55 INFO mapreduce.Job: ?map 45% reduce 0%
14/11/05 18:19:56 INFO mapreduce.Job: ?map 50% reduce 0%
14/11/05 18:20:02 INFO mapreduce.Job: ?map 55% reduce 17%
14/11/05 18:20:03 INFO mapreduce.Job: ?map 59% reduce 17%
14/11/05 18:20:05 INFO mapreduce.Job: ?map 68% reduce 20%
14/11/05 18:20:06 INFO mapreduce.Job: ?map 73% reduce 20%
14/11/05 18:20:08 INFO mapreduce.Job: ?map 73% reduce 24%
14/11/05 18:20:11 INFO mapreduce.Job: ?map 77% reduce 24%
14/11/05 18:20:12 INFO mapreduce.Job: ?map 82% reduce 24%
14/11/05 18:20:13 INFO mapreduce.Job: ?map 91% reduce 24%
14/11/05 18:20:14 INFO mapreduce.Job: ?map 95% reduce 30%
14/11/05 18:20:16 INFO mapreduce.Job: ?map 100% reduce 30%
14/11/05 18:20:17 INFO mapreduce.Job: ?map 100% reduce 100%
14/11/05 18:20:18 INFO mapreduce.Job: Job job_1415182439385_0001 completed successfully
14/11/05 18:20:18 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=54637
FILE: Number of bytes written=2338563
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=59677
HDFS: Number of bytes written=28233
HDFS: Number of read operations=69
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters?
Launched map tasks=22
Launched reduce tasks=1
Data-local map tasks=22
Total time spent by all maps in occupied slots (ms)=185554
Total time spent by all reduces in occupied slots (ms)=30206
Total time spent by all map tasks (ms)=185554
Total time spent by all reduce tasks (ms)=30206
Total vcore-seconds taken by all map tasks=185554
Total vcore-seconds taken by all reduce tasks=30206
Total megabyte-seconds taken by all map tasks=190007296
Total megabyte-seconds taken by all reduce tasks=30930944
Map-Reduce Framework
Map input records=1504
Map output records=5727
Map output bytes=77326
Map output materialized bytes=54763
Input split bytes=2498
Combine input records=5727
Combine output records=2838
Reduce input groups=1224
Reduce shuffle bytes=54763
Reduce input records=2838
Reduce output records=1224
Spilled Records=5676
Shuffled Maps =22
Failed Shuffles=0
Merged Map outputs=22
GC time elapsed (ms)=1707
CPU time spent (ms)=14500
Physical memory (bytes) snapshot=5178937344
Virtual memory (bytes) snapshot=22517506048
Total committed heap usage (bytes)=3882549248
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters?
Bytes Read=57179
File Output Format Counters?
Bytes Written=28233
原文:http://leibnitz.iteye.com/blog/2152751