Modifying VisibilityEvaluator - Problem with Classpath for scanner

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
Hi,

I am trying to make some modifications to the Accumulo VisibilityModifier, but when I add my code, and try to run a scan on a table, I am getting ClassNoDefError for a class that my modified code uses.

So far, what I have been trying to add my JARs to the classpath is to put the JARs in the Accumulo lib/ext directory AND also, editing the accumulo-site.xml and adding the same JARs to that file.  

Is there somewhere else I need to configure or do I need to put the JAR in somewhere other than the lib/ext directory?

Thanks,
Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

ivan bella
Jim,
Putting the jars in the lib/ext directory is a perfectly acceptable mechanism. Make sure your jars are in place on all of your tservers before you start them up. Perhaps this class is being found but is failing to initialize itself which can lead to a ClassNoDefError.
Ivan

> On July 31, 2017 at 2:39 AM o haya <[hidden email]> wrote:
>
>
> Hi,
>
> I am trying to make some modifications to the Accumulo VisibilityModifier, but when I add my code, and try to run a scan on a table, I am getting ClassNoDefError for a class that my modified code uses.
>
> So far, what I have been trying to add my JARs to the classpath is to put the JARs in the Accumulo lib/ext directory AND also, editing the accumulo-site.xml and adding the same JARs to that file.
>
> Is there somewhere else I need to configure or do I need to put the JAR in somewhere other than the lib/ext directory?
>
> Thanks,
> Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

Michael Wall
You can use the classpath command to help troubleshoot as well

$ACCUMULO_HOME/bin/accumulo classpath

On Mon, Jul 31, 2017 at 8:33 AM ivan bella <[hidden email]> wrote:
Jim,
Putting the jars in the lib/ext directory is a perfectly acceptable mechanism. Make sure your jars are in place on all of your tservers before you start them up. Perhaps this class is being found but is failing to initialize itself which can lead to a ClassNoDefError.

Ivan


> On July 31, 2017 at 2:39 AM o haya <[hidden email]> wrote:
>
>
> Hi,
>
> I am trying to make some modifications to the Accumulo VisibilityModifier, but when I add my code, and try to run a scan on a table, I am getting ClassNoDefError for a class that my modified code uses.
>
> So far, what I have been trying to add my JARs to the classpath is to put the JARs in the Accumulo lib/ext directory AND also, editing the accumulo-site.xml and adding the same JARs to that file.
>
> Is there somewhere else I need to configure or do I need to put the JAR in somewhere other than the lib/ext directory?
>
> Thanks,
> Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
In reply to this post by o haya
Hi Michael,

I did use the "accumulo classpath" and it shows all my JARs.  Also during the startup I can see it "Push" each of the JARs, but still get the error.

Sorry - I also just remembered - I output a TON of debug info in my JAR that I added, to stdout, but I don't see any of that debug output in the Accumulo console/stdout or in any of the Accumulo logs.  Is there a way that I can have the debug output from my JAR show up somewhere?  That might help...

Thanks,
Jim
--------------------------------------------
On Mon, 7/31/17, Michael Wall <[hidden email]> wrote:

 Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
 To: [hidden email], "o haya" <[hidden email]>
 Date: Monday, July 31, 2017, 8:34 AM
 
 You can
 use the classpath command to help troubleshoot as
 well
 $ACCUMULO_HOME/bin/accumulo
 classpath
 On Mon, Jul
 31, 2017 at 8:33 AM ivan bella <[hidden email]>
 wrote:
 
 
     
 Jim,
    Putting the jars in the lib/ext
 directory is a perfectly acceptable mechanism.  Make sure
 your jars are in place on all of your tservers before you
 start them up.  Perhaps this class is being found but is
 failing to initialize itself which can lead to a
 ClassNoDefError.
 Ivan
 
 > On July 31, 2017 at 2:39 AM o haya <[hidden email]>
 wrote:
 >
 >
 > Hi,
 >
 > I am trying to make some modifications to
 the Accumulo VisibilityModifier, but when I add my code, and
 try to run a scan on a table, I am getting ClassNoDefError
 for a class that my modified code uses.
 >
 
 > So far, what I have been trying to add
 my JARs to the classpath is to put the JARs in the Accumulo
 lib/ext directory AND also, editing the accumulo-site.xml
 and adding the same JARs to that file.  
 >
 > Is there somewhere
 else I need to configure or do I need to put the JAR in
 somewhere other than the lib/ext directory?
 >
 > Thanks,
 > Jim
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
In reply to this post by o haya
[Sorry for the re-send, but I think I mistakenly sent to only Ivan before.]

Hi Ivan,

My Accumulo configuration is with everything on a single machine.  Would that matter (would I have more than one tserver?)?

My situation is like "whack-a-mole" (that's the game where you hit a mole with a hammer and another one pops up).  What I mean is it seems like I restart Accumulo (bin/stop-all.sh then bin/start-all.sh) and one time I get one class that is not found, then the next time it seems like a different class is not found.  Really frustrating.

So far, as I said, I have put the JARs in lib/ext AND added them to the accumulo-site.xml :(...  I gave up last night and shut all down and will try again later today.

Thanks,
Jim


--------------------------------------------
On Mon, 7/31/17, ivan bella <[hidden email]> wrote:

 Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
 To: [hidden email], "o haya" <[hidden email]>
 Date: Monday, July 31, 2017, 8:33 AM
 
 Jim,
    Putting the jars in the lib/ext directory is
 a perfectly acceptable mechanism.  Make sure your jars are
 in place on all of your tservers before you start them up.
 Perhaps this class is being found but is failing to
 initialize itself which can lead to a ClassNoDefError.
 Ivan
 
 > On July
 31, 2017 at 2:39 AM o haya <[hidden email]> wrote:
 >
 >
 > Hi,
 >
 > I am trying to make some modifications to
 the Accumulo VisibilityModifier, but when I add my code, and
 try to run a scan on a table, I am getting ClassNoDefError
 for a class that my modified code uses.
 >
 
 > So far, what I have been trying to add
 my JARs to the classpath is to put the JARs in the Accumulo
 lib/ext directory AND also, editing the accumulo-site.xml
 and adding the same JARs to that file.  
 >
 > Is there somewhere
 else I need to configure or do I need to put the JAR in
 somewhere other than the lib/ext directory?
 >
 > Thanks,
 > Jim
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

ivan bella
In reply to this post by o haya

Jim,

  All on one machine implies that you only have one $ACCUMULO_HOME/lib/ext directory.  It would not matter if you are running multiple tservers on that machine.  As Mike suggested, using $ACCUMULO_HOME/bin/accumulo classpath would help eliminate whether this is a classpath issue vs a class initialization issue.  Another possible source of issues would be if you have multiple of the same class on the classpath (different jars with the same class).

Regards,

   Ivan


On July 31, 2017 at 10:30 AM o haya <[hidden email]> wrote:

Hi Ivan,

My Accumulo configuration is with everything on a single machine. Would that matter (would I have more than one tserver?)?

My situation is like "whack-a-mole" (that's the game where you hit a mole with a hammer and another one pops up). What I mean is it seems like I restart Accumulo (bin/stop-all.sh then bin/start-all.sh) and one time I get one class that is not found, then the next time it seems like a different class is not found. Really frustrating.

So far, as I said, I have put the JARs in lib/ext AND added them to the accumulo-site.xml :(... I gave up last night and shut all down and will try again later today.

Thanks,
Jim

--------------------------------------------
On Mon, 7/31/17, ivan bella <[hidden email]> wrote:

Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
To: [hidden email], "o haya" <[hidden email]>
Date: Monday, July 31, 2017, 8:33 AM

Jim,
Putting the jars in the lib/ext directory is
a perfectly acceptable mechanism. Make sure your jars are
in place on all of your tservers before you start them up.
Perhaps this class is being found but is failing to
initialize itself which can lead to a ClassNoDefError.
Ivan

On July
31, 2017 at 2:39 AM o haya <[hidden email]> wrote:

Hi,

I am trying to make some modifications to
the Accumulo VisibilityModifier, but when I add my code, and
try to run a scan on a table, I am getting ClassNoDefError
for a class that my modified code uses.

So far, what I have been trying to add
my JARs to the classpath is to put the JARs in the Accumulo
lib/ext directory AND also, editing the accumulo-site.xml
and adding the same JARs to that file.

Is there somewhere
else I need to configure or do I need to put the JAR in
somewhere other than the lib/ext directory?

Thanks,
Jim

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
In reply to this post by o haya
Hi,

Thanks.

I was asking on the other reply about trying to get output from my JARs.  

The reason I was asking that is that I have a suspicion that the not found errors I am seeing may be kind of false indication of what is wrong, i.e., there is some other error or errors, but I am not seeing the output from my JARs, so those errors are hidden, but the not found errors I am seeing is the "visible" output.


--------------------------------------------
On Mon, 7/31/17, ivan bella <[hidden email]> wrote:

 Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
 To: "o haya" <[hidden email]>, [hidden email]
 Date: Monday, July 31, 2017, 10:42 AM
 
 Jim,
   All on one machine implies that you
 only have one $ACCUMULO_HOME/lib/ext directory.  It would
 not matter if you are running multiple tservers on that
 machine.  As Mike suggested, using
 $ACCUMULO_HOME/bin/accumulo classpath would help eliminate
 whether this is a classpath issue vs a class initialization
 issue.  Another possible source of issues would be if you
 have multiple of the same class on the classpath (different
 jars with the same class).
 Regards,
   
 Ivan
 
 On July 31, 2017 at 10:30 AM
 o haya <[hidden email]> wrote:Hi
 Ivan,My Accumulo configuration is with everything on
 a single machine. Would that matter (would I have more than
 one tserver?)?My situation is like
 "whack-a-mole" (that's the game where you hit
 a mole with a hammer and another one pops up). What I mean
 is it seems like I restart Accumulo (bin/stop-all.sh then
 bin/start-all.sh) and one time I get one class that is not
 found, then the next time it seems like a different class is
 not found. Really frustrating.So far, as I said, I
 have put the JARs in lib/ext AND added them to the
 accumulo-site.xml :(... I gave up last night and shut all
 down and will try again later today.Thanks,
 Jim--------------------------------------------
 On Mon, 7/31/17, ivan bella
 <[hidden email]> wrote:Subject: Re:
 Modifying VisibilityEvaluator - Problem with Classpath for
 scanner
  To: [hidden email],
 "o haya" <[hidden email]>
 
 Date: Monday, July 31, 2017, 8:33 AMJim,
  Putting the jars in the lib/ext directory
 is
  a perfectly acceptable mechanism. Make
 sure your jars are
  in place on all of your
 tservers before you start them up.
  Perhaps
 this class is being found but is failing to
 
 initialize itself which can lead to a ClassNoDefError.
  IvanOn July
  31, 2017 at 2:39 AM o haya
 <[hidden email]> wrote:Hi,I am trying
 to make some modifications to
  the Accumulo
 VisibilityModifier, but when I add my code, and
  try to run a scan on a table, I am getting
 ClassNoDefError
  for a class that my
 modified code uses.So far, what I have been trying to
 add
  my JARs to the classpath is to put the
 JARs in the Accumulo
  lib/ext directory AND
 also, editing the accumulo-site.xml
  and
 adding the same JARs to that file.Is there
 somewhere
  else I need to configure or do I
 need to put the JAR in
  somewhere other than
 the lib/ext directory?Thanks,
 Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

Josh Elser
Want to share the log files if you're worried that you're missing
something? I can assure you that Accumulo is not "suppressing real
Exceptions" and giving you "fake" ones.

Also, yes. Accumulo is horizontally scalable. You can run one to many
TabletServers.

On 7/31/17 10:54 AM, o haya wrote:

> Hi,
>
> Thanks.
>
> I was asking on the other reply about trying to get output from my JARs.
>
> The reason I was asking that is that I have a suspicion that the not found errors I am seeing may be kind of false indication of what is wrong, i.e., there is some other error or errors, but I am not seeing the output from my JARs, so those errors are hidden, but the not found errors I am seeing is the "visible" output.
>
>
> --------------------------------------------
> On Mon, 7/31/17, ivan bella <[hidden email]> wrote:
>
>   Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
>   To: "o haya" <[hidden email]>, [hidden email]
>   Date: Monday, July 31, 2017, 10:42 AM
>  
>   Jim,
>     All on one machine implies that you
>   only have one $ACCUMULO_HOME/lib/ext directory.  It would
>   not matter if you are running multiple tservers on that
>   machine.  As Mike suggested, using
>   $ACCUMULO_HOME/bin/accumulo classpath would help eliminate
>   whether this is a classpath issue vs a class initialization
>   issue.  Another possible source of issues would be if you
>   have multiple of the same class on the classpath (different
>   jars with the same class).
>   Regards,
>    
>   Ivan
>  
>   On July 31, 2017 at 10:30 AM
>   o haya <[hidden email]> wrote:Hi
>   Ivan,My Accumulo configuration is with everything on
>   a single machine. Would that matter (would I have more than
>   one tserver?)?My situation is like
>   "whack-a-mole" (that's the game where you hit
>   a mole with a hammer and another one pops up). What I mean
>   is it seems like I restart Accumulo (bin/stop-all.sh then
>   bin/start-all.sh) and one time I get one class that is not
>   found, then the next time it seems like a different class is
>   not found. Really frustrating.So far, as I said, I
>   have put the JARs in lib/ext AND added them to the
>   accumulo-site.xml :(... I gave up last night and shut all
>   down and will try again later today.Thanks,
>   Jim--------------------------------------------
>   On Mon, 7/31/17, ivan bella
>   <[hidden email]> wrote:Subject: Re:
>   Modifying VisibilityEvaluator - Problem with Classpath for
>   scanner
>    To: [hidden email],
>   "o haya" <[hidden email]>
>  
>   Date: Monday, July 31, 2017, 8:33 AMJim,
>    Putting the jars in the lib/ext directory
>   is
>    a perfectly acceptable mechanism. Make
>   sure your jars are
>    in place on all of your
>   tservers before you start them up.
>    Perhaps
>   this class is being found but is failing to
>  
>   initialize itself which can lead to a ClassNoDefError.
>    IvanOn July
>    31, 2017 at 2:39 AM o haya
>   <[hidden email]> wrote:Hi,I am trying
>   to make some modifications to
>    the Accumulo
>   VisibilityModifier, but when I add my code, and
>    try to run a scan on a table, I am getting
>   ClassNoDefError
>    for a class that my
>   modified code uses.So far, what I have been trying to
>   add
>    my JARs to the classpath is to put the
>   JARs in the Accumulo
>    lib/ext directory AND
>   also, editing the accumulo-site.xml
>    and
>   adding the same JARs to that file.Is there
>   somewhere
>    else I need to configure or do I
>   need to put the JAR in
>    somewhere other than
>   the lib/ext directory?Thanks,
>   Jim
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

ivan bella
In reply to this post by o haya

I would expect that all output going to stdout/stderr is being captured in the tserver*.out or tserver*.err files (or master equivalents).  Of course if a class is not being loaded then it does not get much of a chance to output anything....

On July 31, 2017 at 10:34 AM o haya <[hidden email]> wrote:

Hi Michael,

I did use the "accumulo classpath" and it shows all my JARs. Also during the startup I can see it "Push" each of the JARs, but still get the error.

Sorry - I also just remembered - I output a TON of debug info in my JAR that I added, to stdout, but I don't see any of that debug output in the Accumulo console/stdout or in any of the Accumulo logs. Is there a way that I can have the debug output from my JAR show up somewhere? That might help...

Thanks,

Jim

--------------------------------------------
On Mon, 7/31/17, Michael Wall <[hidden email]> wrote:

Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
To: [hidden email], "o haya" <[hidden email]>
Date: Monday, July 31, 2017, 8:34 AM

You can
use the classpath command to help troubleshoot as
well
$ACCUMULO_HOME/bin/accumulo
classpath
On Mon, Jul
31, 2017 at 8:33 AM ivan bella <[hidden email]>
wrote:

Jim,
Putting the jars in the lib/ext
directory is a perfectly acceptable mechanism. Make sure
your jars are in place on all of your tservers before you
start them up. Perhaps this class is being found but is
failing to initialize itself which can lead to a
ClassNoDefError.
Ivan

On July 31, 2017 at 2:39 AM o haya <[hidden email]>
wrote:

Hi,

I am trying to make some modifications to
the Accumulo VisibilityModifier, but when I add my code, and
try to run a scan on a table, I am getting ClassNoDefError
for a class that my modified code uses.

So far, what I have been trying to add
my JARs to the classpath is to put the JARs in the Accumulo
lib/ext directory AND also, editing the accumulo-site.xml
and adding the same JARs to that file.

Is there somewhere
else I need to configure or do I need to put the JAR in
somewhere other than the lib/ext directory?

Thanks,
Jim

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
In reply to this post by o haya
Josh and Ivan,

I don't have access to my test environment now, but FYI, from the stacktraces that I saw, I can see that it actually got quite a bit into my code, and there was a lot of output (to stdout) that was not in the Accumulo logs or in the Accumulo console output.  I will check later today about the tserver files u mentioned but I don't remember iseeing any .out or .err files in the logs directory?  Do the .err and .out files show up in a different directory than the Accumulo logs directory?



--------------------------------------------
On Mon, 7/31/17, ivan bella <[hidden email]> wrote:

 Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
 To: [hidden email]
 Date: Monday, July 31, 2017, 11:55 AM
 
 I would
 expect that all output going to stdout/stderr is being
 captured in the tserver*.out or tserver*.err files (or
 master equivalents).  Of course if a class is not being
 loaded then it does not get much of a chance to output
 anything....
 On July 31, 2017 at 10:34 AM
 o haya <[hidden email]> wrote:Hi
 Michael,I did use the "accumulo classpath"
 and it shows all my JARs. Also during the startup I can see
 it "Push" each of the JARs, but still get the
 error.Sorry - I also just remembered - I output a TON
 of debug info in my JAR that I added, to stdout, but I
 don't see any of that debug output in the Accumulo
 console/stdout or in any of the Accumulo logs. Is there a
 way that I can have the debug output from my JAR show up
 somewhere? That might
 help...Thanks,Jim--------------------------------------------
 On Mon, 7/31/17, Michael Wall
 <[hidden email]> wrote:Subject: Re: Modifying
 VisibilityEvaluator - Problem with Classpath for scanner
  To: [hidden email], "o
 haya" <[hidden email]>
  Date:
 Monday, July 31, 2017, 8:34 AMYou can
  use the classpath command to help troubleshoot
 as
  well
 
 $ACCUMULO_HOME/bin/accumulo
  classpath
  On Mon, Jul
  31, 2017 at 8:33
 AM ivan bella <[hidden email]>
 
 wrote:Jim,
  Putting the jars in the
 lib/ext
  directory is a perfectly acceptable
 mechanism. Make sure
  your jars are in place
 on all of your tservers before you
  start
 them up. Perhaps this class is being found but is
  failing to initialize itself which can lead to
 a
  ClassNoDefError.
 
 IvanOn July 31, 2017 at 2:39 AM o haya
 <[hidden email]>
 
 wrote:Hi,I am trying to make some
 modifications to
  the Accumulo
 VisibilityModifier, but when I add my code, and
  try to run a scan on a table, I am getting
 ClassNoDefError
  for a class that my
 modified code uses.So far, what I have been trying to
 add
  my JARs to the classpath is to put the
 JARs in the Accumulo
  lib/ext directory AND
 also, editing the accumulo-site.xml
  and
 adding the same JARs to that file.Is there
 somewhere
  else I need to configure or do I
 need to put the JAR in
  somewhere other than
 the lib/ext directory?Thanks,
 Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
In reply to this post by o haya
Hi,

Ok, I checked and there are .err and .trc files.  

The .err files are empty, but I do see the stdout output from my classes in the tserver_localhost.whatever.com.out.

So, it definitely is getting into my code, but it looks like any errors are not showing up in the .out file.

Jim

--------------------------------------------
On Mon, 7/31/17, o haya <[hidden email]> wrote:

 Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
 To: [hidden email]
 Date: Monday, July 31, 2017, 12:17 PM
 
 Josh and Ivan,
 
 I don't have access to my test
 environment now, but FYI, from the stacktraces that I saw, I
 can see that it actually got quite a bit into my code, and
 there was a lot of output (to stdout) that was not in the
 Accumulo logs or in the Accumulo console output.  I
 will check later today about the tserver files u mentioned
 but I don't remember iseeing any .out or .err files in the
 logs directory?  Do the .err and .out files show up in
 a different directory than the Accumulo logs directory?
 
 
 
 --------------------------------------------
 On Mon, 7/31/17, ivan bella <[hidden email]>
 wrote:
 
  Subject: Re: Modifying
 VisibilityEvaluator - Problem with Classpath for scanner
  To: [hidden email]
  Date: Monday, July 31, 2017, 11:55 AM
 
  I would
  expect that all output going to
 stdout/stderr is being
  captured in the tserver*.out or
 tserver*.err files (or
  master equivalents).  Of course if a
 class is not being
  loaded then it does not get much of a
 chance to output
  anything....
  On July 31, 2017 at 10:34 AM
  o haya <[hidden email]>
 wrote:Hi
  Michael,I did use the "accumulo
 classpath"
  and it shows all my JARs. Also during
 the startup I can see
  it "Push" each of the JARs, but still
 get the
  error.Sorry - I also just remembered -
 I output a TON
  of debug info in my JAR that I added,
 to stdout, but I
  don't see any of that debug output in
 the Accumulo
  console/stdout or in any of the
 Accumulo logs. Is there a
  way that I can have the debug output
 from my JAR show up
  somewhere? That might
 
 help...Thanks,Jim--------------------------------------------
  On Mon, 7/31/17, Michael Wall
  <[hidden email]>
 wrote:Subject: Re: Modifying
  VisibilityEvaluator - Problem with
 Classpath for scanner
   To: [hidden email],
 "o
  haya" <[hidden email]>
   Date:
  Monday, July 31, 2017, 8:34 AMYou can
   use the classpath command to
 help troubleshoot
  as
   well
 
  $ACCUMULO_HOME/bin/accumulo
   classpath
   On Mon, Jul
   31, 2017 at 8:33
  AM ivan bella <[hidden email]>
 
  wrote:Jim,
   Putting the jars in the
  lib/ext
   directory is a perfectly
 acceptable
  mechanism. Make sure
   your jars are in place
  on all of your tservers before you
   start
  them up. Perhaps this class is being
 found but is
   failing to initialize itself
 which can lead to
  a
   ClassNoDefError.
 
  IvanOn July 31, 2017 at 2:39 AM o
 haya
  <[hidden email]>
 
  wrote:Hi,I am trying to make some
  modifications to
   the Accumulo
  VisibilityModifier, but when I add my
 code, and
   try to run a scan on a table, I
 am getting
  ClassNoDefError
   for a class that my
  modified code uses.So far, what I have
 been trying to
  add
   my JARs to the classpath is to
 put the
  JARs in the Accumulo
   lib/ext directory AND
  also, editing the accumulo-site.xml
   and
  adding the same JARs to that file.Is
 there
  somewhere
   else I need to configure or do
 I
  need to put the JAR in
   somewhere other than
  the lib/ext directory?Thanks,
  Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

o haya
In reply to this post by o haya
Hi,

I have a method in one of my classes that dumps out the system properties, and that includes the Java path and it is showing this:

In listSystemProperties: [00001] java.class.path: [/apps/accumulo-1.8.1/conf:/apps/accumulo-1.8.1/lib/accumulo-start.jar:/apps/accumulo-1.8.1/lib/slf4j-api.jar:/apps/accumulo-1.8.1/lib/slf4j-log4j12.jar:/apps/hadoop-2.7.3//share/hadoop/common/lib/log4j-1.2.17.jar:]

I think that that is a (maybe 'the') problem, i.e., the classpath when it executes my code only has the Accumulo/hadoop JARs and the classpath doesn't include any of the JARs that my code requires.

Is there a way that I can configure Accumulo so that when my Java code is executed, that the classpath can include JARs that my code requires?


Also, I think there is another problem.  The place where it is blowing up is in my code, and it's inside a try-catch.  And inside the catch, I output some error messages to stdout, but that output does not appear in any of the logs including the trc and err files.  Why is that?  And can I get the output of the catch to output somewhere?

Thanks,
Jim



--------------------------------------------
On Mon, 7/31/17, o haya <[hidden email]> wrote:

 Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
 To: [hidden email], "o haya" <[hidden email]>
 Date: Monday, July 31, 2017, 2:04 PM
 
 Hi,
 
 Ok, I checked and there are .err and
 .trc files. 
 
 The .err files are empty, but I do see
 the stdout output from my classes in the
 tserver_localhost.whatever.com.out.
 
 So, it definitely is getting into my
 code, but it looks like any errors are not showing up in the
 .out file.
 
 Jim
 
 --------------------------------------------
 On Mon, 7/31/17, o haya <[hidden email]>
 wrote:
 
  Subject: Re: Modifying
 VisibilityEvaluator - Problem with Classpath for scanner
  To: [hidden email]
  Date: Monday, July 31, 2017, 12:17 PM
 
  Josh and Ivan,
 
  I don't have access to my test
  environment now, but FYI, from the
 stacktraces that I saw, I
  can see that it actually got quite a
 bit into my code, and
  there was a lot of output (to stdout)
 that was not in the
  Accumulo logs or in the Accumulo
 console output.  I
  will check later today about the
 tserver files u mentioned
  but I don't remember iseeing any .out
 or .err files in the
  logs directory?  Do the .err and .out
 files show up in
  a different directory than the
 Accumulo logs directory?
 
 
 
 
 --------------------------------------------
  On Mon, 7/31/17, ivan bella <[hidden email]>
  wrote:
 
   Subject: Re: Modifying
  VisibilityEvaluator - Problem with
 Classpath for scanner
   To: [hidden email]
   Date: Monday, July 31, 2017,
 11:55 AM
  
   I would
   expect that all output going to
  stdout/stderr is being
   captured in the tserver*.out or
  tserver*.err files (or
   master equivalents).  Of course
 if a
  class is not being
   loaded then it does not get much
 of a
  chance to output
   anything....
   On July 31, 2017 at 10:34 AM
   o haya <[hidden email]>
  wrote:Hi
   Michael,I did use the "accumulo
  classpath"
   and it shows all my JARs. Also
 during
  the startup I can see
   it "Push" each of the JARs, but
 still
  get the
   error.Sorry - I also just
 remembered -
  I output a TON
   of debug info in my JAR that I
 added,
  to stdout, but I
   don't see any of that debug
 output in
  the Accumulo
   console/stdout or in any of the
  Accumulo logs. Is there a
   way that I can have the debug
 output
  from my JAR show up
   somewhere? That might
 
 
 help...Thanks,Jim--------------------------------------------
   On Mon, 7/31/17, Michael Wall
   <[hidden email]>
  wrote:Subject: Re: Modifying
   VisibilityEvaluator - Problem
 with
  Classpath for scanner
    To: [hidden email],
  "o
   haya" <[hidden email]>
    Date:
   Monday, July 31, 2017, 8:34
 AMYou can
    use the classpath command to
  help troubleshoot
   as
    well
  
   $ACCUMULO_HOME/bin/accumulo
    classpath
    On Mon, Jul
    31, 2017 at 8:33
   AM ivan bella <[hidden email]>
  
   wrote:Jim,
    Putting the jars in the
   lib/ext
    directory is a perfectly
  acceptable
   mechanism. Make sure
    your jars are in place
   on all of your tservers before
 you
    start
   them up. Perhaps this class is
 being
  found but is
    failing to initialize itself
  which can lead to
   a
    ClassNoDefError.
  
   IvanOn July 31, 2017 at 2:39 AM
 o
  haya
   <[hidden email]>
  
   wrote:Hi,I am trying to make
 some
   modifications to
    the Accumulo
   VisibilityModifier, but when I
 add my
  code, and
    try to run a scan on a table, I
  am getting
   ClassNoDefError
    for a class that my
   modified code uses.So far, what
 I have
  been trying to
   add
    my JARs to the classpath is to
  put the
   JARs in the Accumulo
    lib/ext directory AND
   also, editing the
 accumulo-site.xml
    and
   adding the same JARs to that
 file.Is
  there
   somewhere
    else I need to configure or do
  I
   need to put the JAR in
    somewhere other than
   the lib/ext directory?Thanks,
   Jim
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner

Josh Elser
Accumulo will not load the jars you placed in lib/ or lib/ext via
java.class.path. All you need to do is place the jars in one of those
directories (and restart Accumulo if in lib/). The default
general.classpaths regex in accumulo-site.xml will load all jar files
that are in that directory.

To inspect output from code you're running into Accumulo, you should use
a logger, not stdout/stderr. Accumulo uses log4j and slf4j -- you can
use either of their APIs to accomplish this.

In a real system, this will place output in the .log or .debug.log
files, per the system's log4j configuration in $ACCUMULO_CONF_DIR.
stdout/stderr will be places into the .out and .err files, respectively.

On 7/31/17 6:11 PM, o haya wrote:

> Hi,
>
> I have a method in one of my classes that dumps out the system properties, and that includes the Java path and it is showing this:
>
> In listSystemProperties: [00001] java.class.path: [/apps/accumulo-1.8.1/conf:/apps/accumulo-1.8.1/lib/accumulo-start.jar:/apps/accumulo-1.8.1/lib/slf4j-api.jar:/apps/accumulo-1.8.1/lib/slf4j-log4j12.jar:/apps/hadoop-2.7.3//share/hadoop/common/lib/log4j-1.2.17.jar:]
>
> I think that that is a (maybe 'the') problem, i.e., the classpath when it executes my code only has the Accumulo/hadoop JARs and the classpath doesn't include any of the JARs that my code requires.
>
> Is there a way that I can configure Accumulo so that when my Java code is executed, that the classpath can include JARs that my code requires?
>
>
> Also, I think there is another problem.  The place where it is blowing up is in my code, and it's inside a try-catch.  And inside the catch, I output some error messages to stdout, but that output does not appear in any of the logs including the trc and err files.  Why is that?  And can I get the output of the catch to output somewhere?
>
> Thanks,
> Jim
>
>
>
> --------------------------------------------
> On Mon, 7/31/17, o haya <[hidden email]> wrote:
>
>   Subject: Re: Modifying VisibilityEvaluator - Problem with Classpath for scanner
>   To: [hidden email], "o haya" <[hidden email]>
>   Date: Monday, July 31, 2017, 2:04 PM
>  
>   Hi,
>  
>   Ok, I checked and there are .err and
>   .trc files.
>  
>   The .err files are empty, but I do see
>   the stdout output from my classes in the
>   tserver_localhost.whatever.com.out.
>  
>   So, it definitely is getting into my
>   code, but it looks like any errors are not showing up in the
>   .out file.
>  
>   Jim
>  
>   --------------------------------------------
>   On Mon, 7/31/17, o haya <[hidden email]>
>   wrote:
>  
>    Subject: Re: Modifying
>   VisibilityEvaluator - Problem with Classpath for scanner
>    To: [hidden email]
>    Date: Monday, July 31, 2017, 12:17 PM
>    
>    Josh and Ivan,
>    
>    I don't have access to my test
>    environment now, but FYI, from the
>   stacktraces that I saw, I
>    can see that it actually got quite a
>   bit into my code, and
>    there was a lot of output (to stdout)
>   that was not in the
>    Accumulo logs or in the Accumulo
>   console output.  I
>    will check later today about the
>   tserver files u mentioned
>    but I don't remember iseeing any .out
>   or .err files in the
>    logs directory?  Do the .err and .out
>   files show up in
>    a different directory than the
>   Accumulo logs directory?
>    
>    
>    
>  
>   --------------------------------------------
>    On Mon, 7/31/17, ivan bella <[hidden email]>
>    wrote:
>    
>     Subject: Re: Modifying
>    VisibilityEvaluator - Problem with
>   Classpath for scanner
>     To: [hidden email]
>     Date: Monday, July 31, 2017,
>   11:55 AM
>    
>     I would
>     expect that all output going to
>    stdout/stderr is being
>     captured in the tserver*.out or
>    tserver*.err files (or
>     master equivalents).  Of course
>   if a
>    class is not being
>     loaded then it does not get much
>   of a
>    chance to output
>     anything....
>     On July 31, 2017 at 10:34 AM
>     o haya <[hidden email]>
>    wrote:Hi
>     Michael,I did use the "accumulo
>    classpath"
>     and it shows all my JARs. Also
>   during
>    the startup I can see
>     it "Push" each of the JARs, but
>   still
>    get the
>     error.Sorry - I also just
>   remembered -
>    I output a TON
>     of debug info in my JAR that I
>   added,
>    to stdout, but I
>     don't see any of that debug
>   output in
>    the Accumulo
>     console/stdout or in any of the
>    Accumulo logs. Is there a
>     way that I can have the debug
>   output
>    from my JAR show up
>     somewhere? That might
>    
>  
>   help...Thanks,Jim--------------------------------------------
>     On Mon, 7/31/17, Michael Wall
>     <[hidden email]>
>    wrote:Subject: Re: Modifying
>     VisibilityEvaluator - Problem
>   with
>    Classpath for scanner
>      To: [hidden email],
>    "o
>     haya" <[hidden email]>
>      Date:
>     Monday, July 31, 2017, 8:34
>   AMYou can
>      use the classpath command to
>    help troubleshoot
>     as
>      well
>    
>     $ACCUMULO_HOME/bin/accumulo
>      classpath
>      On Mon, Jul
>      31, 2017 at 8:33
>     AM ivan bella <[hidden email]>
>    
>     wrote:Jim,
>      Putting the jars in the
>     lib/ext
>      directory is a perfectly
>    acceptable
>     mechanism. Make sure
>      your jars are in place
>     on all of your tservers before
>   you
>      start
>     them up. Perhaps this class is
>   being
>    found but is
>      failing to initialize itself
>    which can lead to
>     a
>      ClassNoDefError.
>    
>     IvanOn July 31, 2017 at 2:39 AM
>   o
>    haya
>     <[hidden email]>
>    
>     wrote:Hi,I am trying to make
>   some
>     modifications to
>      the Accumulo
>     VisibilityModifier, but when I
>   add my
>    code, and
>      try to run a scan on a table, I
>    am getting
>     ClassNoDefError
>      for a class that my
>     modified code uses.So far, what
>   I have
>    been trying to
>     add
>      my JARs to the classpath is to
>    put the
>     JARs in the Accumulo
>      lib/ext directory AND
>     also, editing the
>   accumulo-site.xml
>      and
>     adding the same JARs to that
>   file.Is
>    there
>     somewhere
>      else I need to configure or do
>    I
>     need to put the JAR in
>      somewhere other than
>     the lib/ext directory?Thanks,
>     Jim
>
Loading...