Linux使用Eclipse编写WordCount时没有out结果

news/2024/5/20 5:09:36 标签: hadoop, mapreduce, hdfs

创作不易,转载请注明出处

文章目录

  • 一、报错信息
  • 二、原因分析
  • 三、解决方法
  • 另附


一、报错信息

2020-12-05 18:22:17,680 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-12-05 18:22:18,426 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1129)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2020-12-05 18:22:18,428 INFO  jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2020-12-05 18:22:18,896 WARN  mapreduce.JobResourceUploader (JobResourceUploader.java:uploadFiles(64)) - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2020-12-05 18:22:18,902 WARN  mapreduce.JobResourceUploader (JobResourceUploader.java:uploadFiles(171)) - No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2020-12-05 18:22:18,945 INFO  input.FileInputFormat (FileInputFormat.java:listStatus(281)) - Total input paths to process : 1
2020-12-05 18:22:19,208 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(199)) - number of splits:1
2020-12-05 18:22:19,376 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(288)) - Submitting tokens for job: job_local1865594548_0001
2020-12-05 18:22:19,577 INFO  mapreduce.Job (Job.java:submit(1301)) - The url to track the job: http://localhost:8080/
2020-12-05 18:22:19,577 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1346)) - Running job: job_local1865594548_0001
2020-12-05 18:22:19,582 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2020-12-05 18:22:19,588 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2020-12-05 18:22:19,664 WARN  mapred.LocalJobRunner (LocalJobRunner.java:run(560)) - job_local1865594548_0001
org.apache.hadoop.security.AccessControlException: Permission denied: user=master, access=WRITE, inode="/mymapreduce":root:supergroup:drwxr-xr-x
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6547)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6529)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6481)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4290)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4260)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4233)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:853)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:600)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:975)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2036)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2034)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
	at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2744)
	at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2713)
	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:870)
	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:866)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:866)
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:859)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1819)
	at org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.setupJob(FileOutputCommitter.java:291)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:511)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=master, access=WRITE, inode="/mymapreduce":root:supergroup:drwxr-xr-x
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6547)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6529)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6481)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4290)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4260)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4233)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:853)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:600)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:975)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2036)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2034)

	at org.apache.hadoop.ipc.Client.call(Client.java:1470)
	at org.apache.hadoop.ipc.Client.call(Client.java:1401)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
	at com.sun.proxy.$Proxy9.mkdirs(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:539)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy10.mkdirs(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2742)
	... 9 more
2020-12-05 18:22:20,580 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1367)) - Job job_local1865594548_0001 running in uber mode : false
2020-12-05 18:22:20,582 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1374)) -  map 0% reduce 0%
2020-12-05 18:22:20,583 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1387)) - Job job_local1865594548_0001 failed with state FAILED due to: NA
2020-12-05 18:22:20,671 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1392)) - Counters: 0

二、原因分析

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=master, access=WRITE, inode="/mymapreduce":root:supergroup:drwxr-xr-x

由于在HDFS上创建的文件(夹)没有write权限。

三、解决方法

在命令行输入

hadoop fs -chmod 777 /mymapreduce

来修改/mymapreduce文件(夹)的权限,然后再运行,就不会报错了。

另附

在这里插入图片描述

如果没有前面所述log日志报错信息的输出,请查看是否有log4j.properties文件,如果没有,在你自己的package下创建一个,再次运行就可以查看日志了。
log4j.properties文件内容:

log4j.rootLogger=info,appender
log4j.appender.appender=org.apache.log4j.ConsoleAppender
log4j.appender.appender.layout=org.apache.log4j.TTCCLayout

在这里插入图片描述
谢谢朋友们!谢谢朋友们
在这里插入图片描述
在这里插入图片描述


http://www.niftyadmin.cn/n/1346846.html

相关文章

【Python从入门到精通】(七)Python字典(dict)让人人都能找到自己的另一半(键值对,成双成对)

您好,我是码农飞哥(wei158556),感谢您阅读本文,欢迎一键三连哦。 💪🏻 1. Python基础专栏,基础知识一网打尽,9.9元买不了吃亏,买不了上当。 Python从入门到精通 ❤️ 2.网上关于Python的面试练习网站很少,这里给大家推荐一款非常棒的Python题库,从入门到大厂面试…

【Python】(六)Python数据类型-列表和元组,九浅一深,用得到

您好&#xff0c;我是码农飞哥&#xff0c;感谢您阅读本文&#xff0c;欢迎一键三连哦。本文分十个章节介绍数据类型中的列表(list)和元组(tuple)&#xff0c;从使用说到底层实现&#xff0c;包您满意 干货满满&#xff0c;建议收藏&#xff0c;需要用到时常看看。 小伙伴们如有…

Flask框架+ajax框架

** Flask框架 ** # 导入相关的包 from flask import Flask from flask_cors import CORS import random# app是Flask的实例&#xff0c;它接收包或者模块的名字作为参数&#xff0c;但一般都是传递__name__ app Flask(__name__) # 设置作用域 CORS(app)# 对应ajax中url路径…

【Python从入门到精通】(八)Python中的set集合(每个人都是唯一的个体)

您好,我是码农飞哥,感谢您阅读本文,欢迎一键三连哦。 本文主要介绍Python数据类型中的集合(set和frozenset),咱们一次把集合聊个透彻。 干货满满,建议收藏,需要用到时常看看。 小伙伴们如有问题及需要欢迎私信或者留言给我哦~ ~ ~。 文章目录 集合是什么?set集合怎么用…

使用ffmpeg进行视频合并时几个问题记录 Operation not permitted

您好&#xff0c;我是码农飞哥&#xff0c;感谢您阅读本文&#xff0c;欢迎一键三连哦。 这是一篇简单的问题记录文。 干货满满&#xff0c;建议收藏&#xff0c;需要用到时常看看。 小伙伴们如有问题及需要&#xff0c;欢迎踊跃留言哦~ ~ ~。 文章目录视频合并的实现问题1问题…

【Python】(八)Python中的set集合(每个人都是唯一的个体)

您好&#xff0c;我是码农飞哥&#xff0c;感谢您阅读本文&#xff0c;欢迎一键三连哦。本文主要介绍Python数据类型中的集合(set和frozenset)&#xff0c;咱们一次把集合聊个透彻。 干货满满&#xff0c;建议收藏&#xff0c;需要用到时常看看。 小伙伴们如有问题及需要欢迎私…

修改IIS虚拟目录名称

1 echo off2 3 echo ------------------------------------------------------------------------------4 echo -----------------------修改IIS虚拟目录名称------------------------------------5 echo -------------------------------------------------------------------…

php跳转页面

1.header(location:url); header函数前面不能有输出 ! 可以先输出到缓存. 2js echo "<script languagejavascript>"; echo " location网址 "; echo "</script>"; js跳转没有header的条件限制转载于:https://www.cnblogs.com/jinshu…