[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
GitHub user keith-turner opened a pull request:

    https://github.com/apache/accumulo/pull/229

    ACCUMULO-4597 fixed bug in rfile-info

   

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/keith-turner/accumulo ACCUMULO-4597

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/accumulo/pull/229.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #229
   
----
commit de80cf53b66f58a9a99d8a9dd54b5a716fa8df44
Author: Keith Turner <[hidden email]>
Date:   2017-03-06T23:48:29Z

    ACCUMULO-4597 fixed bug in rfile-info

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
Github user asfgit closed the pull request at:

    https://github.com/apache/accumulo/pull/229


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user phrocker commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104804673
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
    Is there any other useful information that could be provided? For example, would it make sense to print out the rest of the locality group column families and help identify if others are listed as a the default and have no column families?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user phrocker commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104804870
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
    haha and it was merged!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user keith-turner commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104807544
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
    You can thank findbugs for the message.  I had some different code that accomplished the same thing and made findbugs unhappy.  So I changed it to this to soothe findbugs.  I don't expect this case to happen, and if it does I would have been happy with an NPE.  There are many places that could have an NPE when there is a bug, we don't handle them all with detailed error messages.
   
    Findbugs has found some nasty bugs.   The false positives can be a little annoying, but I think its worth it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user ctubbsii commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104833643
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
    > Findbugs has found some nasty bugs. The false positives can be a little annoying, but I think its worth it.
   
    +1000


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user phrocker commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104833952
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
     "There are many places that could have an NPE when there is a bug, we don't handle them all with detailed error messages."
    https://s-media-cache-ak0.pinimg.com/736x/88/15/5e/88155efa961948d0627e61236c7d1f2b.jpg


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user keith-turner commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104850830
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
    @phrocker did you clear that meme with @mjwall


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GitHub] accumulo pull request #229: ACCUMULO-4597 fixed bug in rfile-info

joshelser
In reply to this post by joshelser
Github user mjwall commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/229#discussion_r104904485
 
    --- Diff: core/src/main/java/org/apache/accumulo/core/file/rfile/RFile.java ---
    @@ -1015,14 +1018,23 @@ public void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> op
     
         }
     
    +    /**
    +     * @return map of locality group names to column families. The default locality group will have {@code null} for a name. RFile will only track up to
    +     *         {@value Writer#MAX_CF_IN_DLG} families for the default locality group. After this it will stop tracking. For the case where the default group has
    +     *         more thn {@value Writer#MAX_CF_IN_DLG} families an empty list of families is returned.
    +     * @see LocalityGroupUtil#seek(Reader, Range, String, Map)
    +     */
         public Map<String,ArrayList<ByteSequence>> getLocalityGroupCF() {
           Map<String,ArrayList<ByteSequence>> cf = new HashMap<>();
     
           for (LocalityGroupMetadata lcg : localityGroups) {
    -        ArrayList<ByteSequence> setCF = new ArrayList<>();
    +        ArrayList<ByteSequence> setCF;
     
    -        for (Entry<ByteSequence,MutableLong> entry : lcg.columnFamilies.entrySet()) {
    -          setCF.add(entry.getKey());
    +        if (lcg.columnFamilies == null) {
    +          Preconditions.checkState(lcg.isDefaultLG, " Group %s has null families. Only expect default locality group to have null families.", lcg.name);
    --- End diff --
   
    +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---
Loading...