Add a custom timeout for log replica test

[LysandreJik]

Add a custom timeout for log replica test. Let's keep these outliers to a minimum.

[sgugger]

Agreed on trying to keep those to a minimum.

[stas00]

Hmm, it's really slow - clocked 1m16.735s on my machine.

Let me see first if it can be made faster.

[stas00]

It's like 4 tests in one - so it adds up - I guess I could just split it in several sub-tests.

[stas00]

What do you guys prefer here? We can also make it @slow - which will shave off ~80sec - it doesn't need to run all the time at all.

[sgugger]

No strong opinion on my side, do what you think is best!

[stas00]

oh, but these are multi-gpu tests so they are @slow already as they only run on our machine only

@LysandreJik, does this impact the push workflow? or just scheduled one?

If so I'd also @slow all the fairscale/apex tests, as these definitely don't need to run often at all.

[stas00]

But the review is that it should not timeout. As it's 4 tests in one.

Let's decide how to change the test so that it is not an outlier.

[stas00]

Here we go: #12710 - reworked 1 to 4 subtests, shouldn't run longer than the timeout now.

[stas00]

merged the alternative solution, closing this then

[LysandreJik]

The multi GPU tests are run every time there is a commit on master, so it's not only slow tests. We have fast and slow GPU & multi-GPU tests.

Thanks a lot for splitting that test, way better solution.

[stas00]

ok, so should we put fairscale and apex tests to @slow then? These are hardly ever used by anyone, so would be a waste to spend $$ and time on those.

[LysandreJik]

We can, but it's not a super high priority, they seem to run quickly enough