[DISCUSS] Guava Dependencies

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Guava Dependencies

Mike Miller-2
Recently tickets have been opened dealing with Guava in Accumulo (see
ACCUMULO-4701 through 4704), in particular the use of Beta classes and
methods.  Use of Guava comes with a few warnings...

From the Guava README:

*1. APIs marked with the @Beta annotation at the class or method level are
subject to change. They can be modified in any way, or even removed, at any
time. If your code is a library itself (i.e. it is used on the CLASSPATH of
users outside your own control), you should not use beta APIs, unless you
repackage them (e.g. using ProGuard).2.Deprecated non-beta APIs will be
removed two years after the release in which they are first deprecated. You
must fix your references before this time. If you don't, any manner of
breakage could result (you are not guaranteed a compilation error).*

I think it is worth a discussion on how to handle Guava dependencies going
forward across the different versions of Accumulo.  The goal would be to
allow use of a newer version version of Guava in client applications with
the current supported versions of Accumulo.

Ideally, we could just eliminate any use of Beta Guava code.  But there are
Beta classes that are very useful and some which we already have integrated
into released Accumulo versions.

There seem to be 3 ways to handle Guava dependencies:
1 - jar shading
2 - copy Guava code into Accumulo
3 - replace Guava code with standard Java

We may have to handle it differently with each version of Accumulo.  For
example, 1.8 has more widespread use of Beta annotated code than 1.7.
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Guava Dependencies

dlmarion
We still have to use a Hadoop-compatible version of Guava on the server-side though, right? I believe the DFSClient has Guava dependencies.


>
>     On September 18, 2017 at 2:12 PM Mike Miller <[hidden email]> wrote:
>
>     Recently tickets have been opened dealing with Guava in Accumulo (see
>     ACCUMULO-4701 through 4704), in particular the use of Beta classes and
>     methods. Use of Guava comes with a few warnings...
>
>     From the Guava README:
>
>     *1. APIs marked with the @Beta annotation at the class or method level are
>     subject to change. They can be modified in any way, or even removed, at any
>     time. If your code is a library itself (i.e. it is used on the CLASSPATH of
>     users outside your own control), you should not use beta APIs, unless you
>     repackage them (e.g. using ProGuard).2.Deprecated non-beta APIs will be
>     removed two years after the release in which they are first deprecated. You
>     must fix your references before this time. If you don't, any manner of
>     breakage could result (you are not guaranteed a compilation error).*
>
>     I think it is worth a discussion on how to handle Guava dependencies going
>     forward across the different versions of Accumulo. The goal would be to
>     allow use of a newer version version of Guava in client applications with
>     the current supported versions of Accumulo.
>
>     Ideally, we could just eliminate any use of Beta Guava code. But there are
>     Beta classes that are very useful and some which we already have integrated
>     into released Accumulo versions.
>
>     There seem to be 3 ways to handle Guava dependencies:
>     1 - jar shading
>     2 - copy Guava code into Accumulo
>     3 - replace Guava code with standard Java
>
>     We may have to handle it differently with each version of Accumulo. For
>     example, 1.8 has more widespread use of Beta annotated code than 1.7.
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Guava Dependencies

Mike Miller
Right.  We are tied down to Guava 14 because of Hadoop.  This is the work
around since we can't just update Guava.  Previously, we had just avoided
using 14 code that wasn't around anymore.  Which was OK until code was
deprecated after version 14 and then removed, like in the case with
ACCUMULO-4701.  https://issues.apache.org/jira/browse/ACCUMULO-4701

I guess another option would be to do nothing since I believe I have fixed
the problem that was reported in ACCUMULO-4701.

On Mon, Sep 18, 2017 at 2:34 PM, Dave Marion <[hidden email]> wrote:

> We still have to use a Hadoop-compatible version of Guava on the
> server-side though, right? I believe the DFSClient has Guava dependencies.
>
>
> >
> >     On September 18, 2017 at 2:12 PM Mike Miller <[hidden email]>
> wrote:
> >
> >     Recently tickets have been opened dealing with Guava in Accumulo (see
> >     ACCUMULO-4701 through 4704), in particular the use of Beta classes
> and
> >     methods. Use of Guava comes with a few warnings...
> >
> >     From the Guava README:
> >
> >     *1. APIs marked with the @Beta annotation at the class or method
> level are
> >     subject to change. They can be modified in any way, or even removed,
> at any
> >     time. If your code is a library itself (i.e. it is used on the
> CLASSPATH of
> >     users outside your own control), you should not use beta APIs,
> unless you
> >     repackage them (e.g. using ProGuard).2.Deprecated non-beta APIs will
> be
> >     removed two years after the release in which they are first
> deprecated. You
> >     must fix your references before this time. If you don't, any manner
> of
> >     breakage could result (you are not guaranteed a compilation error).*
> >
> >     I think it is worth a discussion on how to handle Guava dependencies
> going
> >     forward across the different versions of Accumulo. The goal would be
> to
> >     allow use of a newer version version of Guava in client applications
> with
> >     the current supported versions of Accumulo.
> >
> >     Ideally, we could just eliminate any use of Beta Guava code. But
> there are
> >     Beta classes that are very useful and some which we already have
> integrated
> >     into released Accumulo versions.
> >
> >     There seem to be 3 ways to handle Guava dependencies:
> >     1 - jar shading
> >     2 - copy Guava code into Accumulo
> >     3 - replace Guava code with standard Java
> >
> >     We may have to handle it differently with each version of Accumulo.
> For
> >     example, 1.8 has more widespread use of Beta annotated code than 1.7.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Guava Dependencies

Josh Elser-2
In reply to this post by Mike Miller-2


On 9/18/17 2:12 PM, Mike Miller wrote:

> Recently tickets have been opened dealing with Guava in Accumulo (see
> ACCUMULO-4701 through 4704), in particular the use of Beta classes and
> methods.  Use of Guava comes with a few warnings...
>
>  From the Guava README:
>
> *1. APIs marked with the @Beta annotation at the class or method level are
> subject to change. They can be modified in any way, or even removed, at any
> time. If your code is a library itself (i.e. it is used on the CLASSPATH of
> users outside your own control), you should not use beta APIs, unless you
> repackage them (e.g. using ProGuard).2.Deprecated non-beta APIs will be
> removed two years after the release in which they are first deprecated. You
> must fix your references before this time. If you don't, any manner of
> breakage could result (you are not guaranteed a compilation error).*
>
> I think it is worth a discussion on how to handle Guava dependencies going
> forward across the different versions of Accumulo.  The goal would be to
> allow use of a newer version version of Guava in client applications with
> the current supported versions of Accumulo.
>
> Ideally, we could just eliminate any use of Beta Guava code.  But there are
> Beta classes that are very useful and some which we already have integrated
> into released Accumulo versions.
>
> There seem to be 3 ways to handle Guava dependencies:
> 1 - jar shading

+1 (favoring #3 too, when not intrusive). We stop "advertising" that we
include Guava on the classpath and it's no longer our problem. As the
other part of the thread alludes, if Hadoop brings in a version, fine.
Accumulo specifically should stop trying to rely on something specific
coming down from its dependencies and "control its own destiny".

FWIW, HBase has been actively moving to this model and, IMO, it's been
working well.

> 2 - copy Guava code into Accumulo
> 3 - replace Guava code with standard Java
>
> We may have to handle it differently with each version of Accumulo.  For
> example, 1.8 has more widespread use of Beta annotated code than 1.7.
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Guava Dependencies

Keith Turner
In reply to this post by Mike Miller
On Mon, Sep 18, 2017 at 2:54 PM, Mike Miller <[hidden email]> wrote:
> Right.  We are tied down to Guava 14 because of Hadoop.  This is the work
> around since we can't just update Guava.  Previously, we had just avoided
> using 14 code that wasn't around anymore.  Which was OK until code was
> deprecated after version 14 and then removed, like in the case with
> ACCUMULO-4701.  https://issues.apache.org/jira/browse/ACCUMULO-4701
>
> I guess another option would be to do nothing since I believe I have fixed
> the problem that was reported in ACCUMULO-4701.

The current Beta classes Accumulo is using (after your PR removes the
one in 4701) still exists in the latest Guava.  We could open an issue
for each one to look into a way to move off of it.  I introduced the
usage for Guava hashing while working on sampling, I didn't realize it
was Beta.  I really wish we had the check for beta in maven at the
time.  If I had known it was beta I would not have used it.  It's hard
to remember to go check the javadocs every time you use Guava.  One
annoying thing about leaving those beta classes is they prevent adding
the automated check for beta to maven.  Ideally we could allow our
current usage of beta classes for the time being and prevent usage of
new beta classes through a maven check.

>
> On Mon, Sep 18, 2017 at 2:34 PM, Dave Marion <[hidden email]> wrote:
>
>> We still have to use a Hadoop-compatible version of Guava on the
>> server-side though, right? I believe the DFSClient has Guava dependencies.
>>
>>
>> >
>> >     On September 18, 2017 at 2:12 PM Mike Miller <[hidden email]>
>> wrote:
>> >
>> >     Recently tickets have been opened dealing with Guava in Accumulo (see
>> >     ACCUMULO-4701 through 4704), in particular the use of Beta classes
>> and
>> >     methods. Use of Guava comes with a few warnings...
>> >
>> >     From the Guava README:
>> >
>> >     *1. APIs marked with the @Beta annotation at the class or method
>> level are
>> >     subject to change. They can be modified in any way, or even removed,
>> at any
>> >     time. If your code is a library itself (i.e. it is used on the
>> CLASSPATH of
>> >     users outside your own control), you should not use beta APIs,
>> unless you
>> >     repackage them (e.g. using ProGuard).2.Deprecated non-beta APIs will
>> be
>> >     removed two years after the release in which they are first
>> deprecated. You
>> >     must fix your references before this time. If you don't, any manner
>> of
>> >     breakage could result (you are not guaranteed a compilation error).*
>> >
>> >     I think it is worth a discussion on how to handle Guava dependencies
>> going
>> >     forward across the different versions of Accumulo. The goal would be
>> to
>> >     allow use of a newer version version of Guava in client applications
>> with
>> >     the current supported versions of Accumulo.
>> >
>> >     Ideally, we could just eliminate any use of Beta Guava code. But
>> there are
>> >     Beta classes that are very useful and some which we already have
>> integrated
>> >     into released Accumulo versions.
>> >
>> >     There seem to be 3 ways to handle Guava dependencies:
>> >     1 - jar shading
>> >     2 - copy Guava code into Accumulo
>> >     3 - replace Guava code with standard Java
>> >
>> >     We may have to handle it differently with each version of Accumulo.
>> For
>> >     example, 1.8 has more widespread use of Beta annotated code than 1.7.
>> >
>>