hbase shell cannot use: ERROR: KeeperErrorCode = NoNode for /hbase/master

Question

I tried to install hbase in pseudo-distributed mode. Now I cannot use hbase now. Every code in the hbase shell cannot run and all of them get this error tip:

ERROR: KeeperErrorCode = NoNode for /hbase/master

In my ubuntu 17 I installed hadoop ,i'm sure that my hdfs location match my hbase:

hdfs://localhost:9000

And this is my hbase-config in hbase-site.xml:

<configuration>
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://localhost:9000/hbase</value>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

My hdfs-config in core-site.xml :

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/usr/local/hadoop/tmp</value>
        <description>A base for other tmp dir</description>
    </property>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

I can start hbase ,but after a while, HMaster gone :

6737 DataNode
7749 HRegionServer
6582 NameNode
6968 SecondaryNameNode
7529 HQuorumPeer
9148 Jps

Log onhttp://localhost:16010/master-status，could see the log:

Failed to become active: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it. (since 2sec ago)

Answer 1

I have also encountered the same problem, while using HBase standalone mode. The jps is able to list HMaster service, but when I start 'hbase shell' and issue any command it throws ERROR: KeeperErrorCode = NoNode for /hbase/master and HMaster service gets terminated abruptly. So using Hbase in pseudo-distributed mode.

I am using -

1: Hadoop - 3.2.0

2: Zookeeper - 3.5.5

3: HBase - 2.2.0

1: I have changed below property in hbase-env.sh, because I want to use separate ZK service instead of embedded one in HBase -

# Tell HBase whether it should manage it's own instance of ZooKeeper or not.
export HBASE_MANAGES_ZK=false

2: Changed hbase-site.xml

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:9000/hbase</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
  </property>
  <property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase</value>
  </property>
</configuration>

3: start-all.sh 4: zkServer.sh start 5: start-hbase.sh

5: jps is able to list HMaster and HRegionServer

madhuri@**-****:$ jps
10688 HRegionServer
4194 DataNode
4019 NameNode
10532 HMaster
4468 SecondaryNameNode
10309 QuorumPeerMain
4902 NodeManager
11162 Main
11740 Jps
4718 ResourceManager
madhuri@**-****:$ 

6:Query HBase :

hbase(main):003:0> list
TABLE                                                                           
mytable                                                                         
1 row(s)
Took 0.0138 seconds                                                             
=> ["mytable"]
hbase(main):004:0> scan 'mytable'
ROW                   COLUMN+CELL                                               
 first                column=cf:message, timestamp=1565095359573, value=hello HB
                      ase                                                       
 second               column=cf:foo, timestamp=1565095375215, value=0           
 third                column=cf:bar, timestamp=1565095394172, value=3.14159     
3 row(s)
Took 0.0186 seconds                                                             
hbase(main):005:0> 

**If everything goes well and still hbase shell takes longer to return result, then please go to ZK diretory - mine is /tmp/zookeeper (Its present in zoo.cfg file) and delete everything and try re-starting above mentioned services. It seems some problem with ZK.

Hope it will help someone!

Answer 2

Make sure you have the following 2 pieces in your hbase-site.xml file. Fixing these solved it for me when I had encountered the same problem:

<configuration>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
  </property>

  <property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase</value>
  </property>
</configuration>

If this still does not fix the problem, watch out for the entries in the hbase log files.

Answer 3

In my case I was receiving this "ERROR: KeeperErrorCode = NoNode for /hbase/master" because HMaster process was not running. As you mentioned your HMaster is also not running.

I don't see hbase.zookeeper.quorum property in your hbase-site.xml. Add it and check.

<property>
<name>hbase.zookeeper.quorum</name>
<value>hdp-master-1</value>
</property>

If still HMaster doesn't start then in $HBASE_HOME/logs directory check for hbase-***-master.log for specific error.

In my case there were 2 reasons ,

First :

WARN  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection timed out

Which I solved by replacing 'localhost' with 'my machine's hostname' in hbase-site.xml. from this answer

Second :

WARN  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection timed out

This was because the hdfs port in hbase-site.xml was different than that in core-site.xml of hadoop.

Answered here.