Need to change location of dataDir in zoo.cfg - but Accumulo doesn't start

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Need to change location of dataDir in zoo.cfg - but Accumulo doesn't start

o haya
Hi,

It's been awhile since I've been working with Accumulo.  To recap I have a standalone system with Hadoop, Zookeeper, and Accumulo 1.8.1 on it.

When I tried to start the Accumulo, it kept looping and indicating "waiting for initialization".

So I was doing some googling and found that the dataDir in the zoo.cfg was set to /tmp.

Luckily (I think) i have a snapshot of this system, so I restored from my last snapshot and it seems to be running all right now, but I want to fix that dataDir parameter, so I created a new directory, /apps/Zookeeper-data, and chmod'ed it to 777, but then, when I try to start Accumulo it just kind of hangs.

I saw from another place that I should run the "accumulo init" so I did that, and now, when I do start-all.sh, it seems like Accumulo is starting, but I cannot get to the Accumulo shell (it starts part wa, then seems to hang, a few lines after I enter the password:

2017-07-29 19:32:49,167 [hdfs.BlockReaderLocal] DEBUG: dfs.client.domain.socket.data.traffic = false
2017-07-29 19:32:49,167 [hdfs.BlockReaderLocal] DEBUG: dfs.domain.socket.path =
2017-07-29 19:32:49,267 [retry.RetryUtils] DEBUG: multipleLinearRandomRetry = null
2017-07-29 19:32:49,306 [ipc.Server] DEBUG: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@15a13207
2017-07-29 19:32:49,319 [ipc.Client] DEBUG: getting client out of cache: org.apache.hadoop.ipc.Client@51de4e31
2017-07-29 19:32:49,767 [unix.DomainSocketWatcher] DEBUG: org.apache.hadoop.net.unix.DomainSocketWatcher$2@16577601: starting with interruptCheckPeriodMs = 60000
2017-07-29 19:32:49,772 [util.PerformanceAdvisory] DEBUG: Both short-circuit local reads and UNIX domain socket are disabled.
2017-07-29 19:32:49,778 [sasl.DataTransferSaslUtil] DEBUG: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
2017-07-29 19:32:49,824 [ipc.Client] DEBUG: The ping interval is 60000 ms.
2017-07-29 19:32:49,826 [ipc.Client] DEBUG: Connecting to localhost/127.0.0.1:9000
2017-07-29 19:32:49,850 [ipc.Client] DEBUG: IPC Client (958411794) connection to localhost/127.0.0.1:9000 from oracle: starting, having connections 1
2017-07-29 19:32:49,861 [ipc.Client] DEBUG: IPC Client (958411794) connection to localhost/127.0.0.1:9000 from oracle sending #0
2017-07-29 19:32:49,870 [ipc.Client] DEBUG: IPC Client (958411794) connection to localhost/127.0.0.1:9000 from oracle got value #0
2017-07-29 19:32:49,870 [ipc.ProtobufRpcEngine] DEBUG: Call: getListing took 61ms
Password: ******
2017-07-29 19:32:54,605 [zookeeper.ZooSession] DEBUG: Connecting to localhost:2181 with timeout 30000 with auth
2017-07-29 19:32:54,814 [tracer.ZooTraceClient] DEBUG: Successfully initialized tracer hosts from ZooKeeper
2017-07-29 19:32:54,815 [trace.DistributedTrace] INFO : SpanReceiver org.apache.accumulo.tracer.ZooTraceClient was loaded successfully.
2017-07-29 19:32:54,843 [impl.ServerClient] WARN : There are no tablet servers: check that zookeeper and accumulo are running.
2017-07-29 19:32:59,851 [ipc.Client] DEBUG: IPC Client (958411794) connection to localhost/127.0.0.1:9000 from oracle: closed
2017-07-29 19:32:59,851 [ipc.Client] DEBUG: IPC Client (958411794) connection to localhost/127.0.0.1:9000 from oracle: stopped, remaining connections 0

I can go back to the snapshot that has the dataDir pointing to /tmp, but can someone tell me the steps that I would need to get the system running  after I change dataDir?

Thanks,
Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Need to change location of dataDir in zoo.cfg - but Accumulo doesn't start

o haya
Hi,

I think it was working all along, but was just taking a really long time to login the first time using shell after init.

Thanks,
Jim

--------------------------------------------
On Sat, 7/29/17, o haya <[hidden email]> wrote:

 Subject: Need to change location of dataDir in zoo.cfg - but Accumulo doesn't start
 To: [hidden email]
 Cc: [hidden email]
 Date: Saturday, July 29, 2017, 7:36 PM
 
 Hi,
 
 It's been awhile since I've been
 working with Accumulo.¬† To recap I have a standalone
 system with Hadoop, Zookeeper, and Accumulo 1.8.1 on it.
 
 When I tried to start the Accumulo, it
 kept looping and indicating "waiting for initialization".
 
 So I was doing some googling and found
 that the dataDir in the zoo.cfg was set to /tmp.
 
 Luckily (I think) i have a snapshot of
 this system, so I restored from my last snapshot and it
 seems to be running all right now, but I want to fix that
 dataDir parameter, so I created a new directory,
 /apps/Zookeeper-data, and chmod'ed it to 777, but then, when
 I try to start Accumulo it just kind of hangs.
 
 I saw from another place that I should
 run the "accumulo init" so I did that, and now, when I do
 start-all.sh, it seems like Accumulo is starting, but I
 cannot get to the Accumulo shell (it starts part wa, then
 seems to hang, a few lines after I enter the password:
 
 2017-07-29 19:32:49,167
 [hdfs.BlockReaderLocal] DEBUG:
 dfs.client.domain.socket.data.traffic = false
 2017-07-29 19:32:49,167
 [hdfs.BlockReaderLocal] DEBUG: dfs.domain.socket.path =
 2017-07-29 19:32:49,267
 [retry.RetryUtils] DEBUG: multipleLinearRandomRetry = null
 2017-07-29 19:32:49,306 [ipc.Server]
 DEBUG: rpcKind=RPC_PROTOCOL_BUFFER,
 rpcRequestWrapperClass=class
 org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper,
 rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@15a13207
 2017-07-29 19:32:49,319 [ipc.Client]
 DEBUG: getting client out of cache: org.apache.hadoop.ipc.Client@51de4e31
 2017-07-29 19:32:49,767
 [unix.DomainSocketWatcher] DEBUG:
 org.apache.hadoop.net.unix.DomainSocketWatcher$2@16577601: starting
 with interruptCheckPeriodMs = 60000
 2017-07-29 19:32:49,772
 [util.PerformanceAdvisory] DEBUG: Both short-circuit local
 reads and UNIX domain socket are disabled.
 2017-07-29 19:32:49,778
 [sasl.DataTransferSaslUtil] DEBUG: DataTransferProtocol not
 using SaslPropertiesResolver, no QOP found in configuration
 for dfs.data.transfer.protection
 2017-07-29 19:32:49,824 [ipc.Client]
 DEBUG: The ping interval is 60000 ms.
 2017-07-29 19:32:49,826 [ipc.Client]
 DEBUG: Connecting to localhost/127.0.0.1:9000
 2017-07-29 19:32:49,850 [ipc.Client]
 DEBUG: IPC Client (958411794) connection to
 localhost/127.0.0.1:9000 from oracle: starting, having
 connections 1
 2017-07-29 19:32:49,861 [ipc.Client]
 DEBUG: IPC Client (958411794) connection to
 localhost/127.0.0.1:9000 from oracle sending #0
 2017-07-29 19:32:49,870 [ipc.Client]
 DEBUG: IPC Client (958411794) connection to
 localhost/127.0.0.1:9000 from oracle got value #0
 2017-07-29 19:32:49,870
 [ipc.ProtobufRpcEngine] DEBUG: Call: getListing took 61ms
 Password: ******
 2017-07-29 19:32:54,605
 [zookeeper.ZooSession] DEBUG: Connecting to localhost:2181
 with timeout 30000 with auth
 2017-07-29 19:32:54,814
 [tracer.ZooTraceClient] DEBUG: Successfully initialized
 tracer hosts from ZooKeeper
 2017-07-29 19:32:54,815
 [trace.DistributedTrace] INFO : SpanReceiver
 org.apache.accumulo.tracer.ZooTraceClient was loaded
 successfully.
 2017-07-29 19:32:54,843
 [impl.ServerClient] WARN : There are no tablet servers:
 check that zookeeper and accumulo are running.
 2017-07-29 19:32:59,851 [ipc.Client]
 DEBUG: IPC Client (958411794) connection to
 localhost/127.0.0.1:9000 from oracle: closed
 2017-07-29 19:32:59,851 [ipc.Client]
 DEBUG: IPC Client (958411794) connection to
 localhost/127.0.0.1:9000 from oracle: stopped, remaining
 connections 0
 
 I can go back to the snapshot that has
 the dataDir pointing to /tmp, but can someone tell me the
 steps that I would need to get the system running¬†
 after I change dataDir?
 
 Thanks,
 Jim
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Need to change location of dataDir in zoo.cfg - but Accumulo doesn't start

Josh Elser
Strange that it would take so long to log in. This is essentially just
an RPC or two. You should be able to open the shell within a few
seconds against an Accumulo instance that is up and running.

To confirm, yes, reinitializing Accumulo is the steps you would need
to take to get Accumulo back up after losing ZooKeeper. Please be
aware for the future that while all of the table data is stored in
HDFS, the table metadata (among other things) are stored in ZooKeeper.
If you lose ZooKeeper and have to reinitiialize, you'll have no
tables. Just a word of caution :)

On Sat, Jul 29, 2017 at 8:38 PM, o haya <[hidden email]> wrote:

> Hi,
>
> I think it was working all along, but was just taking a really long time to login the first time using shell after init.
>
> Thanks,
> Jim
>
> --------------------------------------------
> On Sat, 7/29/17, o haya <[hidden email]> wrote:
>
>  Subject: Need to change location of dataDir in zoo.cfg - but Accumulo doesn't start
>  To: [hidden email]
>  Cc: [hidden email]
>  Date: Saturday, July 29, 2017, 7:36 PM
>
>  Hi,
>
>  It's been awhile since I've been
>  working with Accumulo.  To recap I have a standalone
>  system with Hadoop, Zookeeper, and Accumulo 1.8.1 on it.
>
>  When I tried to start the Accumulo, it
>  kept looping and indicating "waiting for initialization".
>
>  So I was doing some googling and found
>  that the dataDir in the zoo.cfg was set to /tmp.
>
>  Luckily (I think) i have a snapshot of
>  this system, so I restored from my last snapshot and it
>  seems to be running all right now, but I want to fix that
>  dataDir parameter, so I created a new directory,
>  /apps/Zookeeper-data, and chmod'ed it to 777, but then, when
>  I try to start Accumulo it just kind of hangs.
>
>  I saw from another place that I should
>  run the "accumulo init" so I did that, and now, when I do
>  start-all.sh, it seems like Accumulo is starting, but I
>  cannot get to the Accumulo shell (it starts part wa, then
>  seems to hang, a few lines after I enter the password:
>
>  2017-07-29 19:32:49,167
>  [hdfs.BlockReaderLocal] DEBUG:
>  dfs.client.domain.socket.data.traffic = false
>  2017-07-29 19:32:49,167
>  [hdfs.BlockReaderLocal] DEBUG: dfs.domain.socket.path =
>  2017-07-29 19:32:49,267
>  [retry.RetryUtils] DEBUG: multipleLinearRandomRetry = null
>  2017-07-29 19:32:49,306 [ipc.Server]
>  DEBUG: rpcKind=RPC_PROTOCOL_BUFFER,
>  rpcRequestWrapperClass=class
>  org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper,
>  rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@15a13207
>  2017-07-29 19:32:49,319 [ipc.Client]
>  DEBUG: getting client out of cache: org.apache.hadoop.ipc.Client@51de4e31
>  2017-07-29 19:32:49,767
>  [unix.DomainSocketWatcher] DEBUG:
>  org.apache.hadoop.net.unix.DomainSocketWatcher$2@16577601: starting
>  with interruptCheckPeriodMs = 60000
>  2017-07-29 19:32:49,772
>  [util.PerformanceAdvisory] DEBUG: Both short-circuit local
>  reads and UNIX domain socket are disabled.
>  2017-07-29 19:32:49,778
>  [sasl.DataTransferSaslUtil] DEBUG: DataTransferProtocol not
>  using SaslPropertiesResolver, no QOP found in configuration
>  for dfs.data.transfer.protection
>  2017-07-29 19:32:49,824 [ipc.Client]
>  DEBUG: The ping interval is 60000 ms.
>  2017-07-29 19:32:49,826 [ipc.Client]
>  DEBUG: Connecting to localhost/127.0.0.1:9000
>  2017-07-29 19:32:49,850 [ipc.Client]
>  DEBUG: IPC Client (958411794) connection to
>  localhost/127.0.0.1:9000 from oracle: starting, having
>  connections 1
>  2017-07-29 19:32:49,861 [ipc.Client]
>  DEBUG: IPC Client (958411794) connection to
>  localhost/127.0.0.1:9000 from oracle sending #0
>  2017-07-29 19:32:49,870 [ipc.Client]
>  DEBUG: IPC Client (958411794) connection to
>  localhost/127.0.0.1:9000 from oracle got value #0
>  2017-07-29 19:32:49,870
>  [ipc.ProtobufRpcEngine] DEBUG: Call: getListing took 61ms
>  Password: ******
>  2017-07-29 19:32:54,605
>  [zookeeper.ZooSession] DEBUG: Connecting to localhost:2181
>  with timeout 30000 with auth
>  2017-07-29 19:32:54,814
>  [tracer.ZooTraceClient] DEBUG: Successfully initialized
>  tracer hosts from ZooKeeper
>  2017-07-29 19:32:54,815
>  [trace.DistributedTrace] INFO : SpanReceiver
>  org.apache.accumulo.tracer.ZooTraceClient was loaded
>  successfully.
>  2017-07-29 19:32:54,843
>  [impl.ServerClient] WARN : There are no tablet servers:
>  check that zookeeper and accumulo are running.
>  2017-07-29 19:32:59,851 [ipc.Client]
>  DEBUG: IPC Client (958411794) connection to
>  localhost/127.0.0.1:9000 from oracle: closed
>  2017-07-29 19:32:59,851 [ipc.Client]
>  DEBUG: IPC Client (958411794) connection to
>  localhost/127.0.0.1:9000 from oracle: stopped, remaining
>  connections 0
>
>  I can go back to the snapshot that has
>  the dataDir pointing to /tmp, but can someone tell me the
>  steps that I would need to get the system running
>  after I change dataDir?
>
>  Thanks,
>  Jim
>
Loading...