rc2 test question

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

rc2 test question

Owens, Mark
Is anyone having any issues with ShellServerIT failing in the RC2 release?

It is consistently failing for me at line 1691.

assertTrue(trace.contains("sendMutations"));


Log output from test displays 'binMutations' but no 'sendMutations' output.

Trace started at 2019/07/30 15:31:22.336
Time  Start  Service@Location       Name
3417+0      shell@ip-10-113-10-222 shell:root
    2+163      master@0.0.0.0 beginFateOperation
    5+170      master@0.0.0.0 executeFateOperation
    6+177        master@0.0.0.0 CreateTable
    2+183        master@0.0.0.0 CreateTable
   16+187          master@0.0.0.0 SetupPermissions
    3+207            master@0.0.0.0 PopulateZookeeper
   25+210            master@0.0.0.0 PopulateZookeeper
  101+176      master@0.0.0.0 waitForFateOperation
    4+278      master@0.0.0.0 finishFateOperation
1507+288      shell@ip-10-113-10-222 close
   10+288        shell@ip-10-113-10-222 BinMutations 1
    5+288          shell@ip-10-113-10-222 binMutations
    2+290            tserver@0.0.0.0 startScan
    1+292            tserver@0.0.0.0 startScan
    1+292              tserver@0.0.0.0 scan-meta 7
    3+1796     tserver@0.0.0.0 getTableConfiguration
    1+1799     shell@ip-10-113-10-222 client:getTableConfiguration
    3+1800     tserver@0.0.0.0 getTableConfiguration
    3+1804     tserver@0.0.0.0 getTableConfiguration
    3+1808     tserver@0.0.0.0 getTableConfiguration
    8+1814     shell@ip-10-113-10-222 scan
    8+1814       shell@ip-10-113-10-222 scan:location
    8+1814         tserver@0.0.0.0 startScan
    7+1815           tserver@0.0.0.0 scan-default 14
    1+1823     master@0.0.0.0 beginFateOperation
   11+1825     master@0.0.0.0 executeFateOperation
    7+1837       master@0.0.0.0 DeleteTable
    2+1844       master@0.0.0.0 DeleteTable
    3+2294         master@0.0.0.0 CleanUp
    1+2296           master@0.0.0.0 scan
    1+2296             master@0.0.0.0 scan:location
  102+2297         master@0.0.0.0 CleanUp
    1+2310           master@0.0.0.0 batch scanner 659- 1
    2+2311           master@0.0.0.0 scan
    2+2311             master@0.0.0.0 scan:location
    2+2311               tserver@0.0.0.0 startScan
    2+2311                 tserver@0.0.0.0 scan-meta 4
    1+2313           master@0.0.0.0 close
    1+2315           master@0.0.0.0 scan
    1+2315             master@0.0.0.0 scan:location
    1+2315               tserver@0.0.0.0 startScan
  574+1836     master@0.0.0.0 waitForFateOperation
    6+2410     master@0.0.0.0 finishFateOperation

Reply | Threaded
Open this post in threaded view
|

Re: rc2 test question

Keebler, Holly I
Confirmed that this is not failing for me.

________________________________
From: Owens, Mark <[hidden email]>
Sent: Tuesday, July 30, 2019 12:15:59 PM
To: accumulo-dev <[hidden email]>
Subject: rc2 test question

[External Email]

----------------------------------------------------------------------
Is anyone having any issues with ShellServerIT failing in the RC2 release?

It is consistently failing for me at line 1691.

assertTrue(trace.contains("sendMutations"));


Log output from test displays 'binMutations' but no 'sendMutations' output.

Trace started at 2019/07/30 15:31:22.336
Time  Start  Service@Location       Name
3417+0      shell@ip-10-113-10-222 shell:root
    2+163      master@0.0.0.0 beginFateOperation
    5+170      master@0.0.0.0 executeFateOperation
    6+177        master@0.0.0.0 CreateTable
    2+183        master@0.0.0.0 CreateTable
   16+187          master@0.0.0.0 SetupPermissions
    3+207            master@0.0.0.0 PopulateZookeeper
   25+210            master@0.0.0.0 PopulateZookeeper
  101+176      master@0.0.0.0 waitForFateOperation
    4+278      master@0.0.0.0 finishFateOperation
1507+288      shell@ip-10-113-10-222 close
   10+288        shell@ip-10-113-10-222 BinMutations 1
    5+288          shell@ip-10-113-10-222 binMutations
    2+290            tserver@0.0.0.0 startScan
    1+292            tserver@0.0.0.0 startScan
    1+292              tserver@0.0.0.0 scan-meta 7
    3+1796     tserver@0.0.0.0 getTableConfiguration
    1+1799     shell@ip-10-113-10-222 client:getTableConfiguration
    3+1800     tserver@0.0.0.0 getTableConfiguration
    3+1804     tserver@0.0.0.0 getTableConfiguration
    3+1808     tserver@0.0.0.0 getTableConfiguration
    8+1814     shell@ip-10-113-10-222 scan
    8+1814       shell@ip-10-113-10-222 scan:location
    8+1814         tserver@0.0.0.0 startScan
    7+1815           tserver@0.0.0.0 scan-default 14
    1+1823     master@0.0.0.0 beginFateOperation
   11+1825     master@0.0.0.0 executeFateOperation
    7+1837       master@0.0.0.0 DeleteTable
    2+1844       master@0.0.0.0 DeleteTable
    3+2294         master@0.0.0.0 CleanUp
    1+2296           master@0.0.0.0 scan
    1+2296             master@0.0.0.0 scan:location
  102+2297         master@0.0.0.0 CleanUp
    1+2310           master@0.0.0.0 batch scanner 659- 1
    2+2311           master@0.0.0.0 scan
    2+2311             master@0.0.0.0 scan:location
    2+2311               tserver@0.0.0.0 startScan
    2+2311                 tserver@0.0.0.0 scan-meta 4
    1+2313           master@0.0.0.0 close
    1+2315           master@0.0.0.0 scan
    1+2315             master@0.0.0.0 scan:location
    1+2315               tserver@0.0.0.0 startScan
  574+1836     master@0.0.0.0 waitForFateOperation
    6+2410     master@0.0.0.0 finishFateOperation


________________________________

The preceding message (including attachments) is covered by the Electronic Communication Privacy Act, 18 U.S.C. sections 2510-2512, is intended only for the person or entity to which it is addressed, and may contain information that is confidential, protected by attorney-client or other privilege, or otherwise protected from disclosure by law. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error and destroy the original message and all copies.
Reply | Threaded
Open this post in threaded view
|

Re: rc2 test question

Christopher Tubbs-2
That test is a bit non-deterministic. I looked into it awhile back. I
think it has something to do with how the executor service retains a
link to the thread-local parent span in Tracing (or rather, fails to
do so). To fix, we probably need a proper trace-aware thread service,
but that's probably not going to happen any time soon (because it's
not critical, and the future of the tracing libraries are a bit
uncertain at the moment, with HTrace retiring).

In my CI server, I saw ShellServerIT fail for this reason, and then
pass on a subsequent run.

On Tue, Jul 30, 2019 at 1:06 PM Keebler, Holly I
<[hidden email]> wrote:

>
> Confirmed that this is not failing for me.
>
> ________________________________
> From: Owens, Mark <[hidden email]>
> Sent: Tuesday, July 30, 2019 12:15:59 PM
> To: accumulo-dev <[hidden email]>
> Subject: rc2 test question
>
> [External Email]
>
> ----------------------------------------------------------------------
> Is anyone having any issues with ShellServerIT failing in the RC2 release?
>
> It is consistently failing for me at line 1691.
>
> assertTrue(trace.contains("sendMutations"));
>
>
> Log output from test displays 'binMutations' but no 'sendMutations' output.
>
> Trace started at 2019/07/30 15:31:22.336
> Time  Start  Service@Location       Name
> 3417+0      shell@ip-10-113-10-222 shell:root
>     2+163      master@0.0.0.0 beginFateOperation
>     5+170      master@0.0.0.0 executeFateOperation
>     6+177        master@0.0.0.0 CreateTable
>     2+183        master@0.0.0.0 CreateTable
>    16+187          master@0.0.0.0 SetupPermissions
>     3+207            master@0.0.0.0 PopulateZookeeper
>    25+210            master@0.0.0.0 PopulateZookeeper
>   101+176      master@0.0.0.0 waitForFateOperation
>     4+278      master@0.0.0.0 finishFateOperation
> 1507+288      shell@ip-10-113-10-222 close
>    10+288        shell@ip-10-113-10-222 BinMutations 1
>     5+288          shell@ip-10-113-10-222 binMutations
>     2+290            tserver@0.0.0.0 startScan
>     1+292            tserver@0.0.0.0 startScan
>     1+292              tserver@0.0.0.0 scan-meta 7
>     3+1796     tserver@0.0.0.0 getTableConfiguration
>     1+1799     shell@ip-10-113-10-222 client:getTableConfiguration
>     3+1800     tserver@0.0.0.0 getTableConfiguration
>     3+1804     tserver@0.0.0.0 getTableConfiguration
>     3+1808     tserver@0.0.0.0 getTableConfiguration
>     8+1814     shell@ip-10-113-10-222 scan
>     8+1814       shell@ip-10-113-10-222 scan:location
>     8+1814         tserver@0.0.0.0 startScan
>     7+1815           tserver@0.0.0.0 scan-default 14
>     1+1823     master@0.0.0.0 beginFateOperation
>    11+1825     master@0.0.0.0 executeFateOperation
>     7+1837       master@0.0.0.0 DeleteTable
>     2+1844       master@0.0.0.0 DeleteTable
>     3+2294         master@0.0.0.0 CleanUp
>     1+2296           master@0.0.0.0 scan
>     1+2296             master@0.0.0.0 scan:location
>   102+2297         master@0.0.0.0 CleanUp
>     1+2310           master@0.0.0.0 batch scanner 659- 1
>     2+2311           master@0.0.0.0 scan
>     2+2311             master@0.0.0.0 scan:location
>     2+2311               tserver@0.0.0.0 startScan
>     2+2311                 tserver@0.0.0.0 scan-meta 4
>     1+2313           master@0.0.0.0 close
>     1+2315           master@0.0.0.0 scan
>     1+2315             master@0.0.0.0 scan:location
>     1+2315               tserver@0.0.0.0 startScan
>   574+1836     master@0.0.0.0 waitForFateOperation
>     6+2410     master@0.0.0.0 finishFateOperation
>
>
> ________________________________
>
> The preceding message (including attachments) is covered by the Electronic Communication Privacy Act, 18 U.S.C. sections 2510-2512, is intended only for the person or entity to which it is addressed, and may contain information that is confidential, protected by attorney-client or other privilege, or otherwise protected from disclosure by law. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error and destroy the original message and all copies.
Reply | Threaded
Open this post in threaded view
|

Re: rc2 test question

Adam Lerman
In reply to this post by Keebler, Holly I
Mark,

I have seen that test class fail often both on personal machines and AWS
instances.

I can't quantify as I haven't kept track of when it has happened, but I
know it's that class.

Here is a link to some discussion that happened around that class on slack
https://the-asf.slack.com/archives/CERNB8NDC/p1560797596014800?thread_ts=1560797596.014800&cid=CERNB8NDC
Reply | Threaded
Open this post in threaded view
|

Re: rc2 test question

Mike Miller-2
You could run just that IT a few times like so:
mvn clean test -Dit.test=ShellServerIT -Dtest=foo -DfailIfNoTests=false
-Dfailsafe.rerunFailingTestsCount=5

On Tue, Jul 30, 2019 at 1:26 PM Adam Lerman <[hidden email]> wrote:

> Mark,
>
> I have seen that test class fail often both on personal machines and AWS
> instances.
>
> I can't quantify as I haven't kept track of when it has happened, but I
> know it's that class.
>
> Here is a link to some discussion that happened around that class on slack
>
> https://the-asf.slack.com/archives/CERNB8NDC/p1560797596014800?thread_ts=1560797596.014800&cid=CERNB8NDC
>
Reply | Threaded
Open this post in threaded view
|

RE: rc2 test question

Owens, Mark
In reply to this post by Adam Lerman
Yep, after several continual failures it then started passing.

-----Original Message-----
From: Adam Lerman <[hidden email]>
Sent: Tuesday, July 30, 2019 1:26 PM
To: [hidden email]
Subject: Re: rc2 test question

Mark,

I have seen that test class fail often both on personal machines and AWS instances.

I can't quantify as I haven't kept track of when it has happened, but I know it's that class.

Here is a link to some discussion that happened around that class on slack https://the-asf.slack.com/archives/CERNB8NDC/p1560797596014800?thread_ts=1560797596.014800&cid=CERNB8NDC
Reply | Threaded
Open this post in threaded view
|

RE: rc2 test question

Owens, Mark
In reply to this post by Mike Miller-2
I'll give that a try


-----Original Message-----
From: Mike Miller <[hidden email]>
Sent: Tuesday, July 30, 2019 1:37 PM
To: [hidden email]
Subject: Re: rc2 test question

You could run just that IT a few times like so:
mvn clean test -Dit.test=ShellServerIT -Dtest=foo -DfailIfNoTests=false
-Dfailsafe.rerunFailingTestsCount=5

On Tue, Jul 30, 2019 at 1:26 PM Adam Lerman <[hidden email]> wrote:

> Mark,
>
> I have seen that test class fail often both on personal machines and
> AWS instances.
>
> I can't quantify as I haven't kept track of when it has happened, but
> I know it's that class.
>
> Here is a link to some discussion that happened around that class on
> slack
>
> https://the-asf.slack.com/archives/CERNB8NDC/p1560797596014800?thread_
> ts=1560797596.014800&cid=CERNB8NDC
>
Reply | Threaded
Open this post in threaded view
|

Re: rc2 test question

Josh Elser-2
In reply to this post by Owens, Mark
Yeah, this has been sporadically failing since at least 1.7 days.

On 7/30/19 1:37 PM, Owens, Mark wrote:

> Yep, after several continual failures it then started passing.
>
> -----Original Message-----
> From: Adam Lerman <[hidden email]>
> Sent: Tuesday, July 30, 2019 1:26 PM
> To: [hidden email]
> Subject: Re: rc2 test question
>
> Mark,
>
> I have seen that test class fail often both on personal machines and AWS instances.
>
> I can't quantify as I haven't kept track of when it has happened, but I know it's that class.
>
> Here is a link to some discussion that happened around that class on slack https://the-asf.slack.com/archives/CERNB8NDC/p1560797596014800?thread_ts=1560797596.014800&cid=CERNB8NDC
>