[jira] [Updated] (ACCUMULO-391) Multi-table Accumulo input format

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (ACCUMULO-391) Multi-table Accumulo input format

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/ACCUMULO-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

William Slacum updated ACCUMULO-391:
------------------------------------

    Attachment: new-multitable-if.patch

This should be fully fleshed out. I made an attempt to stick closely to the old API in terms of method names, but I made some modifications. Specifically, "per table" settings are encapsulated as map parameters and are only set once. This avoids having to constantly do String appends in the configuration and makes the code a bit simpler.

I tested this by running continuous ingest into two tables and running a simple job (see https://github.com/wjsl/multi-table-if) against the data. I put in an iterator on each table to override the value to ensure that the proper iterator was being applied to the correct table.

I figure there will be more changes requested, so I've marked this patch as not intended for inclusion. If it's acceptable, I'll change the license.
               

> Multi-table Accumulo input format
> ---------------------------------
>
>                 Key: ACCUMULO-391
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-391
>             Project: Accumulo
>          Issue Type: New Feature
>    Affects Versions: 1.5.0-SNAPSHOT
>            Reporter: John Vines
>            Assignee: William Slacum
>            Priority: Minor
>              Labels: mapreduce,
>         Attachments: multi-table-if.patch, new-multitable-if.patch
>
>
> Just realized we had no MR input method which supports multiple Tables for an input format. I would see it making the table the mapper's key and making the Key/Value a tuple, or alternatively have the Table/Key be the key tuple and stick with Values being the value.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira