hbase region中的Client怎么路由到正确的RegionServer
点击联系发帖人
时间：2016-03-20 09:40
2577人阅读
分布式文件系统（14）
近期对hbase多用户插入数据时，regionserver会莫名奇妙的关闭，regionserver的日志有很多异常：
org.apache.hadoop.hbase.DroppedSnapshotException: region: t,02,9.f14b9a1d05aeafb.
& & & & at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1000)
& & & & at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:905)
& & & & at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:857)
& & & & at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:394)
& & & & at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202)
& & & & at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:222)
00:48:27,671 WARN org.apache.hadoop.hbase.regionserver.Store: Failed open of hdfs://cloudgis4:9000/hbase/t/c85d7d3bc3a55a93a147f5c4f07f87b8/imageFamily/fb520c2adca643 presumption is that file was corrupted at
flush and lost edits picked up by commit log replay. Verify!
java.io.IOException: Filesystem closed
& & & & at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:241)
& & & & at org.apache.hadoop.hdfs.DFSClient.access$800(DFSClient.java:74)
& & & & at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2037)
& & & & at java.io.DataInputStream.readFully(DataInputStream.java:178)
& & & & at java.io.DataInputStream.readLong(DataInputStream.java:399)
& & & & at org.apache.hadoop.hbase.io.hfile.HFile$FixedFileTrailer.deserialize(HFile.java:1526)
& & & & at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:885)
& & & & at org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
& & & & at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.loadFileInfo(StoreFile.java:1003)
& & & & at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:382)
& & & & at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:438)
ABORTING region server serverName=cloudgis1,2412016, load=(requests=1662, regions=111, usedHeap=3758, maxHeap=4991): Replay of HLog required. Forcing server shutdown
00:48:20,081 INFO org.apache.hadoop.hdfs.DFSClient: Exception in createBlockOutputStream java.net.SocketException: Too many open files
在网上查了很久也没有解决办法，把日志从头看了一遍，发现一句话：
00:48:16,939 WARN org.apache.hadoop.hdfs.DFSClient: Failed to connect to /192.168.3.6:50010, add to deadNodes and continuejava.net.SocketException: Too many open files
原来linux对打开文件数有限制，而datanode无法打开文件，所以就回报异常，regionserver也关闭了。解决方法如下：
如果你使用的是Ubuntu,你可以这样设置:
在文件&/etc/security/limits.conf&添加一行，如:
可以把&hadoop&替换成你运行Hbase和Hadoop的用户。如果你用两个用户，你就需要配两个。还有配nproc hard 和 soft limits. 如:
hadoop soft/hard nproc 32000
在&/etc/pam.d/common-session&加上这一行:
session required
pam_limits.so
否则在&/etc/security/limits.conf上的配置不会生效.
还有注销再登录，这些配置才能生效!
一个 Hadoop HDFS Datanode 有一个同时处理文件的上限. 这个参数叫&xcievers&(Hadoop的作者把这个单词拼错了). 在你加载之前，先确认下你有没有配置这个文件conf/hdfs-site.xml里面的xceivers参数，至少要有4096:
&property&
&name&dfs.datanode.max.xcievers&/name&
&value&4096&/value&
&/property&
对于HDFS修改配置要记得重启.
如果没有这一项配置，你可能会遇到奇怪的失败。你会在Datanode的日志中看到xcievers exceeded，但是运行起来会报 missing blocks错误。例如:&10/12/08 20:10:31 INFO hdfs.DFSClient: Could not obtain block blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException: No live nodes
contain current block. Will get new block locations from namenode and retry...&[]
* 以上用户言论只代表其个人观点，不代表CSDN网站的观点或立场
访问：171958次
积分：2590
积分：2590
排名：第8653名
原创：90篇
转载：11篇
评论：62条
(1)(1)(3)(2)(3)(4)(3)(8)(1)(4)(38)(19)(2)(4)(4)(1)(1)(2)HBase集群安装过程中的问题集锦 - 李克华 - 博客园
云计算高级群:
交流：Hadoop、NoSQL、分布式、lucene、solr、nutch
1、HRegionServer启动不正常
在namenode上执行jps,则可看到hbase启动是否正常，进程如下：
[root@master bin]# jps26341 HMaster26642 Jps7840 ResourceManager7524 NameNode7699 SecondaryNameNode
由上可见，hadoop启动正常。HBase少了一个进程，猜测应该是有个节点regionserver没有启动成功。
进入节点slave1 ,执行jps查看启动进程：
[root@master bin]# ssh slave1Last login: Thu Jul 17 17:29:11 2014 from master[root@slave1 ~]# jps4296 DataNode11261 HRegionServer11512 Jps11184 QuorumPeerMain
由此可见Slave1节点正常。
进入节点slave2节点，执行jps查看启动进程：
[root@slave2 ~]# jps3795 DataNode11339 Jps11080 QuorumPeerMain
OK，问题找到了 HRegionServer没有启动成功。进入HBase日志：
09:28:19,392 INFO& [regionserver60020] regionserver.HRegionServer: STOPPED: Unhandled: org.apache.hadoop.hbase.ClockOutOfSyncException: Server slave2,0498057 Reported time is too far out of sync with master.& Time difference of ms & max allowed of 30000ms&&&&&&& at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:314)&&&&&&& at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:215)&&&&&&& at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:1292)&&&&&&& at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5085)&&&&&&& at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2185)&&&&&&& at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1889)
根据错误日志，可得到slave2和maste机器时间差太多，查看各个系统的时间，果真如此，同步即可。另外一种方法就是配置hbase的配置文件：
配置：hbase.master.maxclockske
&property&&&&&&&&&&&&&&&& &name&hbase.master.maxclockskew&/name&&&&&&&&&&&&&&&& &value&200000&/value&&&&&&&&&&&&&&&& &description&Time difference of regionserver from master&/description&&&&&&&& &/property&
（这种方法不推荐）
2、Zookeeper启动不正常。
在启动hbase时，总是报错，提示zookeeper连接不上，查看zookeeper日志，发现：
ClientCnxn$SendThread@966] - Opening socket connection to server slave1. Will not attempt to authenticate using SASL (无法定位登录配置)。经过百度可得
由于hosts文件的问题，于是vi /etc/hosts 发现 ip slave1配置中ip错误。汗！幸亏hbase和zookeeper都有日志。于是重启zookeeper和hbase，上述问题解决。
3、HBase shell执行list命令报错。
在Hbase shell执行list命令报错：
14:01:32,384 ERROR [main] zookeeper.ZooKeeperWatcher: hconnection-0x4b936059, quorum=slave2:2181,slave1:2181,master:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:199)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:822)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.access$200(HConnectionManager.java:544)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1517)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1563)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1618)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1826)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.listTableNames(HConnectionManager.java:2542)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTableNames(HConnectionManager.java:2532)
at org.apache.hadoop.hbase.client.HBaseAdmin.getTableNames(HBaseAdmin.java:352)
at org.apache.hadoop.hbase.client.HBaseAdmin.getTableNames(HBaseAdmin.java:368)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:311)
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:59)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:63)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.RubyClass.finvoke(RubyClass.java:552)
at org.jruby.RubyBasicObject.send(RubyBasicObject.java:2777)
at org.jruby.RubyKernel.send(RubyKernel.java:2105)
at org.jruby.RubyKernel$s$send.call(RubyKernel$s$send.gen:65535)
at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrNBlock.call(JavaMethod.java:266)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:39)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
at org.jruby.ast.FCallSpecialArgBlockNode.interpret(FCallSpecialArgBlockNode.java:34)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:215)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
at org.jruby.ast.CallSpecialArgNode.interpret(CallSpecialArgNode.java:69)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:215)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
at org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:41)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallSpecialArgNode.interpret(CallSpecialArgNode.java:67)
at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:147)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:183)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at org.jruby.ast.VCallNode.interpret(VCallNode.java:86)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.RootNode.interpret(RootNode.java:129)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_EVAL(ASTInterpreter.java:95)
at org.jruby.evaluator.ASTInterpreter.evalWithBinding(ASTInterpreter.java:166)
at org.jruby.RubyKernel.evalCommon(RubyKernel.java:1155)
at org.jruby.RubyKernel.eval(RubyKernel.java:1112)
at org.jruby.RubyKernel$s$0$3$eval.call(RubyKernel$s$0$3$eval.gen:65535)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:181)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:282)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:71)
at org.jruby.ast.FCallManyArgsNode.interpret(FCallManyArgsNode.java:60)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:120)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:134)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:174)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:282)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:71)
at org.jruby.ast.CallManyArgsNode.interpret(CallManyArgsNode.java:59)
at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:215)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:212)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:207)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:260)
at org.jruby.runtime.Block.yieldSpecific(Block.java:117)
at org.jruby.ast.YieldTwoNode.interpret(YieldTwoNode.java:31)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.IfNode.interpret(IfNode.java:117)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.RubyKernel.loop(RubyKernel.java:1439)
at org.jruby.RubyKernel$s$0$0$loop.call(RubyKernel$s$0$0$loop.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
at org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:347)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:304)
at org.jruby.runtime.Block.yield(Block.java:130)
at org.jruby.RubyContinuation.enter(RubyContinuation.java:106)
at org.jruby.RubyKernel.rbCatch(RubyKernel.java:1212)
at org.jruby.RubyKernel$s$1$0$rbCatch.call(RubyKernel$s$1$0$rbCatch.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:147)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:183)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at root.hadoop.hbase.bin.hirb.block_2$RUBY$start(/root/hadoop/hbase/bin/hirb.rb:185)
at root$hadoop$hbase$bin$hirb$block_2$RUBY$start.call(root$hadoop$hbase$bin$hirb$block_2$RUBY$start:65535)
at org.piledBlock.yield(CompiledBlock.java:112)
at org.piledBlock.yield(CompiledBlock.java:95)
at org.jruby.runtime.Block.yield(Block.java:130)
at org.jruby.RubyContinuation.enter(RubyContinuation.java:106)
at org.jruby.RubyKernel.rbCatch(RubyKernel.java:1212)
at org.jruby.RubyKernel$s$1$0$rbCatch.call(RubyKernel$s$1$0$rbCatch.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at root.hadoop.hbase.bin.hirb.method__5$RUBY$start(/root/hadoop/hbase/bin/hirb.rb:184)
at root$hadoop$hbase$bin$hirb$method__5$RUBY$start.call(root$hadoop$hbase$bin$hirb$method__5$RUBY$start:65535)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:203)
at org.jruby.internal.piledMethod.call(CompiledMethod.java:255)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at root.hadoop.hbase.bin.hirb.__file__(/root/hadoop/hbase/bin/hirb.rb:190)
at root.hadoop.hbase.bin.hirb.load(/root/hadoop/hbase/bin/hirb.rb)
at org.jruby.Ruby.runScript(Ruby.java:697)
at org.jruby.Ruby.runScript(Ruby.java:690)
at org.jruby.Ruby.runNormally(Ruby.java:597)
at org.jruby.Ruby.runFromMain(Ruby.java:446)
at org.jruby.Main.doRunFromMain(Main.java:369)
at org.jruby.Main.internalRun(Main.java:258)
at org.jruby.Main.run(Main.java:224)
at org.jruby.Main.run(Main.java:208)
at org.jruby.Main.main(Main.java:188)
14:01:32,387 ERROR [main] client.HConnectionManager$HConnectionImplementation: Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase
关键错误信息：client.HConnectionManager$HConnectionImplementation: Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase。根据信息可以判断zk无法连接。执行jps查看zk都正常。查看hbase-site.xml中zk节点配置正常。根据经验，应该是防火墙没有关闭，2181端口无法访问。ok执行service iptables stop关闭防火墙，重启hbase。进入hbase shell，执行list:
hbase(main):001:0& listTABLESLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/root/hadoop/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/root/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See
for an explanation. 14:06:26,013 WARN& [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable0 row(s) in 1.0070 seconds
一切正常，问题解决。
4、HBase Shell 增删改异常。
在hbase shell上做增删改就会报异常，
zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect。经判断是hbase版本的jar包和hadoop中的jar包不兼容的问题。解决方法：将hadoop中hadoop-2.2.0相关的jar包copy过来（${HABASE_HOME}/lib）替换即可。
随笔 - 251}
分布式文件系统（14）
近期对hbase多用户插入数据时，regionserver会莫名奇妙的关闭，regionserver的日志有很多异常：
org.apache.hadoop.hbase.DroppedSnapshotException: region: t,02,9.f14b9a1d05aeafb.
& & & & at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1000)
& & & & at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:905)
& & & & at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:857)
& & & & at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:394)
& & & & at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202)
& & & & at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:222)
00:48:27,671 WARN org.apache.hadoop.hbase.regionserver.Store: Failed open of hdfs://cloudgis4:9000/hbase/t/c85d7d3bc3a55a93a147f5c4f07f87b8/imageFamily/fb520c2adca643 presumption is that file was corrupted at
flush and lost edits picked up by commit log replay. Verify!
java.io.IOException: Filesystem closed
& & & & at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:241)
& & & & at org.apache.hadoop.hdfs.DFSClient.access$800(DFSClient.java:74)
& & & & at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2037)
& & & & at java.io.DataInputStream.readFully(DataInputStream.java:178)
& & & & at java.io.DataInputStream.readLong(DataInputStream.java:399)
& & & & at org.apache.hadoop.hbase.io.hfile.HFile$FixedFileTrailer.deserialize(HFile.java:1526)
& & & & at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:885)
& & & & at org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
& & & & at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.loadFileInfo(StoreFile.java:1003)
& & & & at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:382)
& & & & at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:438)
ABORTING region server serverName=cloudgis1,2412016, load=(requests=1662, regions=111, usedHeap=3758, maxHeap=4991): Replay of HLog required. Forcing server shutdown
00:48:20,081 INFO org.apache.hadoop.hdfs.DFSClient: Exception in createBlockOutputStream java.net.SocketException: Too many open files
在网上查了很久也没有解决办法，把日志从头看了一遍，发现一句话：
00:48:16,939 WARN org.apache.hadoop.hdfs.DFSClient: Failed to connect to /192.168.3.6:50010, add to deadNodes and continuejava.net.SocketException: Too many open files
原来linux对打开文件数有限制，而datanode无法打开文件，所以就回报异常，regionserver也关闭了。解决方法如下：
如果你使用的是Ubuntu,你可以这样设置:
在文件&/etc/security/limits.conf&添加一行，如:
可以把&hadoop&替换成你运行Hbase和Hadoop的用户。如果你用两个用户，你就需要配两个。还有配nproc hard 和 soft limits. 如:
hadoop soft/hard nproc 32000
在&/etc/pam.d/common-session&加上这一行:
session required
pam_limits.so
否则在&/etc/security/limits.conf上的配置不会生效.
还有注销再登录，这些配置才能生效!
一个 Hadoop HDFS Datanode 有一个同时处理文件的上限. 这个参数叫&xcievers&(Hadoop的作者把这个单词拼错了). 在你加载之前，先确认下你有没有配置这个文件conf/hdfs-site.xml里面的xceivers参数，至少要有4096:
&property&
&name&dfs.datanode.max.xcievers&/name&
&value&4096&/value&
&/property&
对于HDFS修改配置要记得重启.
如果没有这一项配置，你可能会遇到奇怪的失败。你会在Datanode的日志中看到xcievers exceeded，但是运行起来会报 missing blocks错误。例如:&10/12/08 20:10:31 INFO hdfs.DFSClient: Could not obtain block blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException: No live nodes
contain current block. Will get new block locations from namenode and retry...&[]
* 以上用户言论只代表其个人观点，不代表CSDN网站的观点或立场
访问：171958次
积分：2590
积分：2590
排名：第8653名
原创：90篇
转载：11篇
评论：62条
(1)(1)(3)(2)(3)(4)(3)(8)(1)(4)(38)(19)(2)(4)(4)(1)(1)(2)HBase集群安装过程中的问题集锦 - 李克华 - 博客园
云计算高级群:
交流：Hadoop、NoSQL、分布式、lucene、solr、nutch
1、HRegionServer启动不正常
在namenode上执行jps,则可看到hbase启动是否正常，进程如下：
[root@master bin]# jps26341 HMaster26642 Jps7840 ResourceManager7524 NameNode7699 SecondaryNameNode
由上可见，hadoop启动正常。HBase少了一个进程，猜测应该是有个节点regionserver没有启动成功。
进入节点slave1 ,执行jps查看启动进程：
[root@master bin]# ssh slave1Last login: Thu Jul 17 17:29:11 2014 from master[root@slave1 ~]# jps4296 DataNode11261 HRegionServer11512 Jps11184 QuorumPeerMain
由此可见Slave1节点正常。
进入节点slave2节点，执行jps查看启动进程：
[root@slave2 ~]# jps3795 DataNode11339 Jps11080 QuorumPeerMain
OK，问题找到了 HRegionServer没有启动成功。进入HBase日志：
09:28:19,392 INFO& [regionserver60020] regionserver.HRegionServer: STOPPED: Unhandled: org.apache.hadoop.hbase.ClockOutOfSyncException: Server slave2,0498057 Reported time is too far out of sync with master.& Time difference of ms & max allowed of 30000ms&&&&&&& at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:314)&&&&&&& at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:215)&&&&&&& at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:1292)&&&&&&& at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5085)&&&&&&& at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2185)&&&&&&& at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1889)
根据错误日志，可得到slave2和maste机器时间差太多，查看各个系统的时间，果真如此，同步即可。另外一种方法就是配置hbase的配置文件：
配置：hbase.master.maxclockske
&property&&&&&&&&&&&&&&&& &name&hbase.master.maxclockskew&/name&&&&&&&&&&&&&&&& &value&200000&/value&&&&&&&&&&&&&&&& &description&Time difference of regionserver from master&/description&&&&&&&& &/property&
（这种方法不推荐）
2、Zookeeper启动不正常。
在启动hbase时，总是报错，提示zookeeper连接不上，查看zookeeper日志，发现：
ClientCnxn$SendThread@966] - Opening socket connection to server slave1. Will not attempt to authenticate using SASL (无法定位登录配置)。经过百度可得
由于hosts文件的问题，于是vi /etc/hosts 发现 ip slave1配置中ip错误。汗！幸亏hbase和zookeeper都有日志。于是重启zookeeper和hbase，上述问题解决。
3、HBase shell执行list命令报错。
在Hbase shell执行list命令报错：
14:01:32,384 ERROR [main] zookeeper.ZooKeeperWatcher: hconnection-0x4b936059, quorum=slave2:2181,slave1:2181,master:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:199)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:822)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.access$200(HConnectionManager.java:544)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1517)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1563)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1618)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1826)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.listTableNames(HConnectionManager.java:2542)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getTableNames(HConnectionManager.java:2532)
at org.apache.hadoop.hbase.client.HBaseAdmin.getTableNames(HBaseAdmin.java:352)
at org.apache.hadoop.hbase.client.HBaseAdmin.getTableNames(HBaseAdmin.java:368)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:311)
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:59)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:63)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.RubyClass.finvoke(RubyClass.java:552)
at org.jruby.RubyBasicObject.send(RubyBasicObject.java:2777)
at org.jruby.RubyKernel.send(RubyKernel.java:2105)
at org.jruby.RubyKernel$s$send.call(RubyKernel$s$send.gen:65535)
at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrNBlock.call(JavaMethod.java:266)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:39)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
at org.jruby.ast.FCallSpecialArgBlockNode.interpret(FCallSpecialArgBlockNode.java:34)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:215)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
at org.jruby.ast.CallSpecialArgNode.interpret(CallSpecialArgNode.java:69)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:215)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
at org.jruby.ast.FCallSpecialArgNode.interpret(FCallSpecialArgNode.java:41)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:190)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:199)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
at org.jruby.ast.CallSpecialArgNode.interpret(CallSpecialArgNode.java:67)
at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:147)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:183)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at org.jruby.ast.VCallNode.interpret(VCallNode.java:86)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.RootNode.interpret(RootNode.java:129)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_EVAL(ASTInterpreter.java:95)
at org.jruby.evaluator.ASTInterpreter.evalWithBinding(ASTInterpreter.java:166)
at org.jruby.RubyKernel.evalCommon(RubyKernel.java:1155)
at org.jruby.RubyKernel.eval(RubyKernel.java:1112)
at org.jruby.RubyKernel$s$0$3$eval.call(RubyKernel$s$0$3$eval.gen:65535)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:181)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:282)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:71)
at org.jruby.ast.FCallManyArgsNode.interpret(FCallManyArgsNode.java:60)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:120)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:134)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:174)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:282)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:71)
at org.jruby.ast.CallManyArgsNode.interpret(CallManyArgsNode.java:59)
at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:233)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:215)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:203)
at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:212)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:207)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:260)
at org.jruby.runtime.Block.yieldSpecific(Block.java:117)
at org.jruby.ast.YieldTwoNode.interpret(YieldTwoNode.java:31)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.IfNode.interpret(IfNode.java:117)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:216)
at org.jruby.ast.RescueNode.interpretWithJavaExceptions(RescueNode.java:120)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:110)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:295)
at org.jruby.runtime.InterpretedBlock.yieldSpecific(InterpretedBlock.java:229)
at org.jruby.runtime.Block.yieldSpecific(Block.java:99)
at org.jruby.RubyKernel.loop(RubyKernel.java:1439)
at org.jruby.RubyKernel$s$0$0$loop.call(RubyKernel$s$0$0$loop.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
at org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:111)
at org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:374)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:347)
at org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:304)
at org.jruby.runtime.Block.yield(Block.java:130)
at org.jruby.RubyContinuation.enter(RubyContinuation.java:106)
at org.jruby.RubyKernel.rbCatch(RubyKernel.java:1212)
at org.jruby.RubyKernel$s$1$0$rbCatch.call(RubyKernel$s$1$0$rbCatch.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:153)
at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:147)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:183)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at root.hadoop.hbase.bin.hirb.block_2$RUBY$start(/root/hadoop/hbase/bin/hirb.rb:185)
at root$hadoop$hbase$bin$hirb$block_2$RUBY$start.call(root$hadoop$hbase$bin$hirb$block_2$RUBY$start:65535)
at org.piledBlock.yield(CompiledBlock.java:112)
at org.piledBlock.yield(CompiledBlock.java:95)
at org.jruby.runtime.Block.yield(Block.java:130)
at org.jruby.RubyContinuation.enter(RubyContinuation.java:106)
at org.jruby.RubyKernel.rbCatch(RubyKernel.java:1212)
at org.jruby.RubyKernel$s$1$0$rbCatch.call(RubyKernel$s$1$0$rbCatch.gen:65535)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:322)
at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:178)
at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:187)
at root.hadoop.hbase.bin.hirb.method__5$RUBY$start(/root/hadoop/hbase/bin/hirb.rb:184)
at root$hadoop$hbase$bin$hirb$method__5$RUBY$start.call(root$hadoop$hbase$bin$hirb$method__5$RUBY$start:65535)
at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:203)
at org.jruby.internal.piledMethod.call(CompiledMethod.java:255)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
at root.hadoop.hbase.bin.hirb.__file__(/root/hadoop/hbase/bin/hirb.rb:190)
at root.hadoop.hbase.bin.hirb.load(/root/hadoop/hbase/bin/hirb.rb)
at org.jruby.Ruby.runScript(Ruby.java:697)
at org.jruby.Ruby.runScript(Ruby.java:690)
at org.jruby.Ruby.runNormally(Ruby.java:597)
at org.jruby.Ruby.runFromMain(Ruby.java:446)
at org.jruby.Main.doRunFromMain(Main.java:369)
at org.jruby.Main.internalRun(Main.java:258)
at org.jruby.Main.run(Main.java:224)
at org.jruby.Main.run(Main.java:208)
at org.jruby.Main.main(Main.java:188)
14:01:32,387 ERROR [main] client.HConnectionManager$HConnectionImplementation: Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase
关键错误信息：client.HConnectionManager$HConnectionImplementation: Can't get connection to ZooKeeper: KeeperErrorCode = ConnectionLoss for /hbase。根据信息可以判断zk无法连接。执行jps查看zk都正常。查看hbase-site.xml中zk节点配置正常。根据经验，应该是防火墙没有关闭，2181端口无法访问。ok执行service iptables stop关闭防火墙，重启hbase。进入hbase shell，执行list:
hbase(main):001:0& listTABLESLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/root/hadoop/hbase/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/root/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See
for an explanation. 14:06:26,013 WARN& [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable0 row(s) in 1.0070 seconds
一切正常，问题解决。
4、HBase Shell 增删改异常。
在hbase shell上做增删改就会报异常，
zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect。经判断是hbase版本的jar包和hadoop中的jar包不兼容的问题。解决方法：将hadoop中hadoop-2.2.0相关的jar包copy过来（${HABASE_HOME}/lib）替换即可。
随笔 - 251}
我要回帖
更多推荐
- ·台湾的微博是什么软件微博和大陆微博一样吗
- ·新移动硬盘显示本地磁盘但无法读取第二次使用“我的电脑”里不显示，“磁盘管理”看到被分成了两个区，是病毒吗？请大神帮忙！
- ·日产蓝鸟空调外循环进风口在哪里如何打开？
- ·本田XR-V好还是启辰大V好1.8L手动和启辰T70 2.0L自动哪个车值得拥有？
- ·A能ping不通的几种可能原因通B，B能ping不通的几种可能原因通A和C，为啥A不能ping不通的几种可能原因通C?
- ·夏普ar2008d原装鼓芯 原装硒鼓多少钱
- ·联想a360t格机发不了短信
- ·6．下面程序的运行结果是_______。 #c main不用includee<stdio.h> int main
- ·乐视年费多少买一年年费有什么优恵
- ·QQ号8位数qq谁买？
- ·唧种mw150u无线网卡驱动支持三星5900三星5900电视。mw300tvmw150u无线网卡驱动怎么连三星5900电观。
- ·求《火锅英雄 网盘》完整版高清百度云网盘下载链接或迅雷下载网址？
- ·电磁场内幕十码十码网站
- ·联想天逸100 14英寸联想笔记本电脑天逸i3-5005u 4g 500g 1g怎样
- ·广发银行信用卡提额的标准是用卡租房都要半年以上吗，那么用卡半年的概念是指从领卡之日算起，还是从卡片激活使用之
- ·麻烦问一下大神们为什么我的百度地图凯立德导航地图下载会这样呢？
- ·怎么解决ubuntu拼音输入法 乱14.04不能打正确拼音
- ·死侍…普罗米修斯1-2…埃及众神战 下载…僵尸时间大战死神来了的百度云网盘？谁有？
- ·求助，UIWebView拖到底部出现的excel删除多余空白列能去掉吗
- ·在喵窝wiki下载的 オトメスイッチ ～彼が持ってる彼女のリモコン～的六个文件压解出来dms和dmf用光驱
- ·hbase region中的Client怎么路由到正确的RegionServer
- ·页面显示已修完四级425能报六级吗达到425是什么意思
- ·沙龙365官网会不会坑人？
- ·亮剑东南东南早报微信公众平台号微信怎么加
- ·设置系统相关属性,实现打开的窗体以重叠任务栏窗口不重叠形式显示
- ·电脑没声音怎么办？装了声卡驱动没声音安装失败怎么解决
- ·联想电脑苹果几天包换换
- ·电脑显示器冒烟了电脑开机后显示器黑屏出现一堆乱码进不去桌面怎么回事
- ·购买北汽绅宝x55试驾视频可以免费获取乐视会员1年吗
- ·徐汇公安局破案为破案还原已删除了的监控是不是费用很高
- ·如何取到uitableview的section自定义的sectionheader
- ·请微视频请人转发朋友圈找谁的软件去哪里买比较靠谱啊？
- ·华为手机在空纸上cad画图实际操作如何操作
- ·求解rays te37 限量版SL和TTA 什么区别 懂的答
- ·遥控邪恶少女漫画id社3d百度云我id一只有尊严的单身狗&#8943;感谢
- ·不用充电器充电也能充电的软件有哪些
