metric gathering

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

metric gathering

Miguel Pereira
Hi all,

Is there an easy way to access Accumulo metrics? I noticed
org.apache.accumulo.server.monitor
is used for the many of the metrics on the monitoring page. Is there any
additional api to support performance gathering? Also what do you guys think
would be important to measure that could support the dev team? My goal is
to use ganglia with accumulo.

Thanks,
Miguel Pereira
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Eric Newton
You have three options:

1) You can turn on JMX monitoring.
2) You can use org.apache.accumulo.server.test.GetMasterStats to print
the data the monitor uses
3) You can pull the xml from the monitor page

-Eric

On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
<[hidden email]> wrote:

> Hi all,
>
> Is there an easy way to access Accumulo metrics? I noticed
> org.apache.accumulo.server.monitor
> is used for the many of the metrics on the monitoring page. Is there any
> additional api to support performance gathering? Also what do you guys think
> would be important to measure that could support the dev team? My goal is
> to use ganglia with accumulo.
>
> Thanks,
> Miguel Pereira
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Miguel Pereira
Thanks Eric!

Are you guys currently using ganglia or is most of your monitoring done
through jmx?

-Miguel

On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <[hidden email]> wrote:

> You have three options:
>
> 1) You can turn on JMX monitoring.
> 2) You can use org.apache.accumulo.server.test.GetMasterStats to print
> the data the monitor uses
> 3) You can pull the xml from the monitor page
>
> -Eric
>
> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
> <[hidden email]> wrote:
> > Hi all,
> >
> > Is there an easy way to access Accumulo metrics? I noticed
> > org.apache.accumulo.server.monitor
> > is used for the many of the metrics on the monitoring page. Is there any
> > additional api to support performance gathering? Also what do you guys
> think
> > would be important to measure that could support the dev team? My goal is
> > to use ganglia with accumulo.
> >
> > Thanks,
> > Miguel Pereira
>
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Eric Newton
I do most of my monitoring (and first-line trouble-shooting) via the
Accumulo monitor page.

I know most teams use XML to import data into their monitoring systems
(based on the complaints I received when I changed the XML).

I did port OpenTSDB to Accumulo; and it reads data via JMX.

I used to work for Zenoss, so I'm partial to it, too.

-Eric

On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
<[hidden email]> wrote:

> Thanks Eric!
>
> Are you guys currently using ganglia or is most of your monitoring done
> through jmx?
>
> -Miguel
>
> On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <[hidden email]> wrote:
>
>> You have three options:
>>
>> 1) You can turn on JMX monitoring.
>> 2) You can use org.apache.accumulo.server.test.GetMasterStats to print
>> the data the monitor uses
>> 3) You can pull the xml from the monitor page
>>
>> -Eric
>>
>> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
>> <[hidden email]> wrote:
>> > Hi all,
>> >
>> > Is there an easy way to access Accumulo metrics? I noticed
>> > org.apache.accumulo.server.monitor
>> > is used for the many of the metrics on the monitoring page. Is there any
>> > additional api to support performance gathering? Also what do you guys
>> think
>> > would be important to measure that could support the dev team? My goal is
>> > to use ganglia with accumulo.
>> >
>> > Thanks,
>> > Miguel Pereira
>>
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Miguel Pereira
I was hoping the accumulo conf dir would have a hadoop-metrics.properties
file similar to hbase were you just drop these lines in..

hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
hbase.period=10
hbase.servers=master.yourcompany.com:8649

jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
jvm.period=10
jvm.servers=master.yourcompany.com:8649

rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
rpc.period=10
rpc.servers=master.yourcompany.com:8649


I did spot the accumulo-metrics.xml and the metrics page in the doc
directory. Anyone have a few examples of how to use JMX here, I feel like i
am really close.

cheers and have a great weekend,
Miguel


On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <[hidden email]> wrote:

> I do most of my monitoring (and first-line trouble-shooting) via the
> Accumulo monitor page.
>
> I know most teams use XML to import data into their monitoring systems
> (based on the complaints I received when I changed the XML).
>
> I did port OpenTSDB to Accumulo; and it reads data via JMX.
>
> I used to work for Zenoss, so I'm partial to it, too.
>
> -Eric
>
> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
> <[hidden email]> wrote:
> > Thanks Eric!
> >
> > Are you guys currently using ganglia or is most of your monitoring done
> > through jmx?
> >
> > -Miguel
> >
> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <[hidden email]>
> wrote:
> >
> >> You have three options:
> >>
> >> 1) You can turn on JMX monitoring.
> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to print
> >> the data the monitor uses
> >> 3) You can pull the xml from the monitor page
> >>
> >> -Eric
> >>
> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
> >> <[hidden email]> wrote:
> >> > Hi all,
> >> >
> >> > Is there an easy way to access Accumulo metrics? I noticed
> >> > org.apache.accumulo.server.monitor
> >> > is used for the many of the metrics on the monitoring page. Is there
> any
> >> > additional api to support performance gathering? Also what do you guys
> >> think
> >> > would be important to measure that could support the dev team? My
> goal is
> >> > to use ganglia with accumulo.
> >> >
> >> > Thanks,
> >> > Miguel Pereira
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Eric Newton
Edit accumulo-metrics.xml, turn the false settings to true, restart,
and then look at the exposed JMX beans with jconsole. You may need to
expose a jmx port with various "-D" options, none of which I remember
how to do.

I don't know much about ganglia.  I've used it a little, but
displaying 100's of graphs on a web page is almost as bad as the the
lack of visualization and historical data in the accumulo monitor.  I
don't know how to configure it.

I didn't find it difficult to modify tcollector, the performance
collector for OpenTSDB, to collect metrics for the Accumulo tservers.

If you do manage it, please post instructions!

-Eric

On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
<[hidden email]> wrote:

> I was hoping the accumulo conf dir would have a hadoop-metrics.properties
> file similar to hbase were you just drop these lines in..
>
> hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> hbase.period=10
> hbase.servers=master.yourcompany.com:8649
>
> jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> jvm.period=10
> jvm.servers=master.yourcompany.com:8649
>
> rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> rpc.period=10
> rpc.servers=master.yourcompany.com:8649
>
>
> I did spot the accumulo-metrics.xml and the metrics page in the doc
> directory. Anyone have a few examples of how to use JMX here, I feel like i
> am really close.
>
> cheers and have a great weekend,
> Miguel
>
>
> On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <[hidden email]> wrote:
>
>> I do most of my monitoring (and first-line trouble-shooting) via the
>> Accumulo monitor page.
>>
>> I know most teams use XML to import data into their monitoring systems
>> (based on the complaints I received when I changed the XML).
>>
>> I did port OpenTSDB to Accumulo; and it reads data via JMX.
>>
>> I used to work for Zenoss, so I'm partial to it, too.
>>
>> -Eric
>>
>> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
>> <[hidden email]> wrote:
>> > Thanks Eric!
>> >
>> > Are you guys currently using ganglia or is most of your monitoring done
>> > through jmx?
>> >
>> > -Miguel
>> >
>> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <[hidden email]>
>> wrote:
>> >
>> >> You have three options:
>> >>
>> >> 1) You can turn on JMX monitoring.
>> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to print
>> >> the data the monitor uses
>> >> 3) You can pull the xml from the monitor page
>> >>
>> >> -Eric
>> >>
>> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
>> >> <[hidden email]> wrote:
>> >> > Hi all,
>> >> >
>> >> > Is there an easy way to access Accumulo metrics? I noticed
>> >> > org.apache.accumulo.server.monitor
>> >> > is used for the many of the metrics on the monitoring page. Is there
>> any
>> >> > additional api to support performance gathering? Also what do you guys
>> >> think
>> >> > would be important to measure that could support the dev team? My
>> goal is
>> >> > to use ganglia with accumulo.
>> >> >
>> >> > Thanks,
>> >> > Miguel Pereira
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Miguel Pereira
Hey guys,

were would I enable a jmx port for accumulo?
the -D options i want to use are

-Dcom.sun.management.jmxremote.port=1105
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

but i don't know were to put them, accumulo-env.sh ?
I plan to use jmxtrans to feed metrics to ganglia & maybe graphite

Cheers,
Miguel

On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <[hidden email]> wrote:

> Edit accumulo-metrics.xml, turn the false settings to true, restart,
> and then look at the exposed JMX beans with jconsole. You may need to
> expose a jmx port with various "-D" options, none of which I remember
> how to do.
>
> I don't know much about ganglia.  I've used it a little, but
> displaying 100's of graphs on a web page is almost as bad as the the
> lack of visualization and historical data in the accumulo monitor.  I
> don't know how to configure it.
>
> I didn't find it difficult to modify tcollector, the performance
> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
>
> If you do manage it, please post instructions!
>
> -Eric
>
> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
> <[hidden email]> wrote:
> > I was hoping the accumulo conf dir would have a hadoop-metrics.properties
> > file similar to hbase were you just drop these lines in..
> >
> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > hbase.period=10
> > hbase.servers=master.yourcompany.com:8649
> >
> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > jvm.period=10
> > jvm.servers=master.yourcompany.com:8649
> >
> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> > rpc.period=10
> > rpc.servers=master.yourcompany.com:8649
> >
> >
> > I did spot the accumulo-metrics.xml and the metrics page in the doc
> > directory. Anyone have a few examples of how to use JMX here, I feel
> like i
> > am really close.
> >
> > cheers and have a great weekend,
> > Miguel
> >
> >
> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <[hidden email]>
> wrote:
> >
> >> I do most of my monitoring (and first-line trouble-shooting) via the
> >> Accumulo monitor page.
> >>
> >> I know most teams use XML to import data into their monitoring systems
> >> (based on the complaints I received when I changed the XML).
> >>
> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
> >>
> >> I used to work for Zenoss, so I'm partial to it, too.
> >>
> >> -Eric
> >>
> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
> >> <[hidden email]> wrote:
> >> > Thanks Eric!
> >> >
> >> > Are you guys currently using ganglia or is most of your monitoring
> done
> >> > through jmx?
> >> >
> >> > -Miguel
> >> >
> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <[hidden email]>
> >> wrote:
> >> >
> >> >> You have three options:
> >> >>
> >> >> 1) You can turn on JMX monitoring.
> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
> print
> >> >> the data the monitor uses
> >> >> 3) You can pull the xml from the monitor page
> >> >>
> >> >> -Eric
> >> >>
> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
> >> >> <[hidden email]> wrote:
> >> >> > Hi all,
> >> >> >
> >> >> > Is there an easy way to access Accumulo metrics? I noticed
> >> >> > org.apache.accumulo.server.monitor
> >> >> > is used for the many of the metrics on the monitoring page. Is
> there
> >> any
> >> >> > additional api to support performance gathering? Also what do you
> guys
> >> >> think
> >> >> > would be important to measure that could support the dev team? My
> >> goal is
> >> >> > to use ganglia with accumulo.
> >> >> >
> >> >> > Thanks,
> >> >> > Miguel Pereira
> >> >>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Eric Newton
You can put them in your accumulo-env.sh.  Be sure to enable the
metrics in the accumulo-metrix.xml file, too.

I did this when I ported OpenTSDB to accumulo.  It worked great.

-Eric

On Fri, Jul 13, 2012 at 3:44 PM, Miguel Pereira
<[hidden email]> wrote:

> Hey guys,
>
> were would I enable a jmx port for accumulo?
> the -D options i want to use are
>
> -Dcom.sun.management.jmxremote.port=1105
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
>
> but i don't know were to put them, accumulo-env.sh ?
> I plan to use jmxtrans to feed metrics to ganglia & maybe graphite
>
> Cheers,
> Miguel
>
> On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <[hidden email]> wrote:
>
>> Edit accumulo-metrics.xml, turn the false settings to true, restart,
>> and then look at the exposed JMX beans with jconsole. You may need to
>> expose a jmx port with various "-D" options, none of which I remember
>> how to do.
>>
>> I don't know much about ganglia.  I've used it a little, but
>> displaying 100's of graphs on a web page is almost as bad as the the
>> lack of visualization and historical data in the accumulo monitor.  I
>> don't know how to configure it.
>>
>> I didn't find it difficult to modify tcollector, the performance
>> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
>>
>> If you do manage it, please post instructions!
>>
>> -Eric
>>
>> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
>> <[hidden email]> wrote:
>> > I was hoping the accumulo conf dir would have a hadoop-metrics.properties
>> > file similar to hbase were you just drop these lines in..
>> >
>> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> > hbase.period=10
>> > hbase.servers=master.yourcompany.com:8649
>> >
>> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> > jvm.period=10
>> > jvm.servers=master.yourcompany.com:8649
>> >
>> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
>> > rpc.period=10
>> > rpc.servers=master.yourcompany.com:8649
>> >
>> >
>> > I did spot the accumulo-metrics.xml and the metrics page in the doc
>> > directory. Anyone have a few examples of how to use JMX here, I feel
>> like i
>> > am really close.
>> >
>> > cheers and have a great weekend,
>> > Miguel
>> >
>> >
>> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <[hidden email]>
>> wrote:
>> >
>> >> I do most of my monitoring (and first-line trouble-shooting) via the
>> >> Accumulo monitor page.
>> >>
>> >> I know most teams use XML to import data into their monitoring systems
>> >> (based on the complaints I received when I changed the XML).
>> >>
>> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
>> >>
>> >> I used to work for Zenoss, so I'm partial to it, too.
>> >>
>> >> -Eric
>> >>
>> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
>> >> <[hidden email]> wrote:
>> >> > Thanks Eric!
>> >> >
>> >> > Are you guys currently using ganglia or is most of your monitoring
>> done
>> >> > through jmx?
>> >> >
>> >> > -Miguel
>> >> >
>> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <[hidden email]>
>> >> wrote:
>> >> >
>> >> >> You have three options:
>> >> >>
>> >> >> 1) You can turn on JMX monitoring.
>> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
>> print
>> >> >> the data the monitor uses
>> >> >> 3) You can pull the xml from the monitor page
>> >> >>
>> >> >> -Eric
>> >> >>
>> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
>> >> >> <[hidden email]> wrote:
>> >> >> > Hi all,
>> >> >> >
>> >> >> > Is there an easy way to access Accumulo metrics? I noticed
>> >> >> > org.apache.accumulo.server.monitor
>> >> >> > is used for the many of the metrics on the monitoring page. Is
>> there
>> >> any
>> >> >> > additional api to support performance gathering? Also what do you
>> guys
>> >> >> think
>> >> >> > would be important to measure that could support the dev team? My
>> >> goal is
>> >> >> > to use ganglia with accumulo.
>> >> >> >
>> >> >> > Thanks,
>> >> >> > Miguel Pereira
>> >> >>
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: metric gathering

Miguel Pereira
Thanks for the help Eric,

I successfully queried accumulo metrics with jmxtrans and passed them to
ganglia.
Going to try it with an actual cluster before posting instructions..

Cheers,
Miguel

On Fri, Jul 13, 2012 at 4:07 PM, Eric Newton <[hidden email]> wrote:

> You can put them in your accumulo-env.sh.  Be sure to enable the
> metrics in the accumulo-metrix.xml file, too.
>
> I did this when I ported OpenTSDB to accumulo.  It worked great.
>
> -Eric
>
> On Fri, Jul 13, 2012 at 3:44 PM, Miguel Pereira
> <[hidden email]> wrote:
> > Hey guys,
> >
> > were would I enable a jmx port for accumulo?
> > the -D options i want to use are
> >
> > -Dcom.sun.management.jmxremote.port=1105
> > -Dcom.sun.management.jmxremote.authenticate=false
> > -Dcom.sun.management.jmxremote.ssl=false
> >
> > but i don't know were to put them, accumulo-env.sh ?
> > I plan to use jmxtrans to feed metrics to ganglia & maybe graphite
> >
> > Cheers,
> > Miguel
> >
> > On Fri, Jun 29, 2012 at 6:52 PM, Eric Newton <[hidden email]>
> wrote:
> >
> >> Edit accumulo-metrics.xml, turn the false settings to true, restart,
> >> and then look at the exposed JMX beans with jconsole. You may need to
> >> expose a jmx port with various "-D" options, none of which I remember
> >> how to do.
> >>
> >> I don't know much about ganglia.  I've used it a little, but
> >> displaying 100's of graphs on a web page is almost as bad as the the
> >> lack of visualization and historical data in the accumulo monitor.  I
> >> don't know how to configure it.
> >>
> >> I didn't find it difficult to modify tcollector, the performance
> >> collector for OpenTSDB, to collect metrics for the Accumulo tservers.
> >>
> >> If you do manage it, please post instructions!
> >>
> >> -Eric
> >>
> >> On Fri, Jun 29, 2012 at 5:07 PM, Miguel Pereira
> >> <[hidden email]> wrote:
> >> > I was hoping the accumulo conf dir would have a
> hadoop-metrics.properties
> >> > file similar to hbase were you just drop these lines in..
> >> >
> >> > hbase.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > hbase.period=10
> >> > hbase.servers=master.yourcompany.com:8649
> >> >
> >> > jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > jvm.period=10
> >> > jvm.servers=master.yourcompany.com:8649
> >> >
> >> > rpc.class=org.apache.hadoop.metrics.ganglia.GangliaContext
> >> > rpc.period=10
> >> > rpc.servers=master.yourcompany.com:8649
> >> >
> >> >
> >> > I did spot the accumulo-metrics.xml and the metrics page in the doc
> >> > directory. Anyone have a few examples of how to use JMX here, I feel
> >> like i
> >> > am really close.
> >> >
> >> > cheers and have a great weekend,
> >> > Miguel
> >> >
> >> >
> >> > On Fri, Jun 29, 2012 at 2:00 PM, Eric Newton <[hidden email]>
> >> wrote:
> >> >
> >> >> I do most of my monitoring (and first-line trouble-shooting) via the
> >> >> Accumulo monitor page.
> >> >>
> >> >> I know most teams use XML to import data into their monitoring
> systems
> >> >> (based on the complaints I received when I changed the XML).
> >> >>
> >> >> I did port OpenTSDB to Accumulo; and it reads data via JMX.
> >> >>
> >> >> I used to work for Zenoss, so I'm partial to it, too.
> >> >>
> >> >> -Eric
> >> >>
> >> >> On Fri, Jun 29, 2012 at 11:55 AM, Miguel Pereira
> >> >> <[hidden email]> wrote:
> >> >> > Thanks Eric!
> >> >> >
> >> >> > Are you guys currently using ganglia or is most of your monitoring
> >> done
> >> >> > through jmx?
> >> >> >
> >> >> > -Miguel
> >> >> >
> >> >> > On Fri, Jun 29, 2012 at 11:09 AM, Eric Newton <
> [hidden email]>
> >> >> wrote:
> >> >> >
> >> >> >> You have three options:
> >> >> >>
> >> >> >> 1) You can turn on JMX monitoring.
> >> >> >> 2) You can use org.apache.accumulo.server.test.GetMasterStats to
> >> print
> >> >> >> the data the monitor uses
> >> >> >> 3) You can pull the xml from the monitor page
> >> >> >>
> >> >> >> -Eric
> >> >> >>
> >> >> >> On Fri, Jun 29, 2012 at 11:03 AM, Miguel Pereira
> >> >> >> <[hidden email]> wrote:
> >> >> >> > Hi all,
> >> >> >> >
> >> >> >> > Is there an easy way to access Accumulo metrics? I noticed
> >> >> >> > org.apache.accumulo.server.monitor
> >> >> >> > is used for the many of the metrics on the monitoring page. Is
> >> there
> >> >> any
> >> >> >> > additional api to support performance gathering? Also what do
> you
> >> guys
> >> >> >> think
> >> >> >> > would be important to measure that could support the dev team?
> My
> >> >> goal is
> >> >> >> > to use ganglia with accumulo.
> >> >> >> >
> >> >> >> > Thanks,
> >> >> >> > Miguel Pereira
> >> >> >>
> >> >>
> >>
>