Re: blocked migrations due to timeout
Hi Andrey,
Thanks for your reply.
> On Apr 24, 2020, at 08:32, Andrey Rahmatullin <wrar@debian.org> wrote:
>
> On Fri, Apr 24, 2020 at 07:29:40AM -0700, Matthew Fernandez wrote:
>> The arm64 failure [2] looks legitimate (though I
>> don’t understand why excuses also says “arch:arm64 not built yet”),
> It says it cannot run autopkgtests for arm64 because the binary isn't
> built yet.
Ah, I see, I don’t get the same warning for mipsel and armel because the binary built fine there. It’s just that the test suite didn’t complete.
>> the mipsel failure [3] looks like some kind of timeout.
> It's a build failure caused by the tests not finishing or producing any
> output, yes.
What is the general approach for long running test suites? Is there a way to extend the timeout? Or is it a hardline that test suites have to complete within the allotted time?
>> The armel build [4] looks like it also timed out, but for reasons I
>> don’t understand this isn’t considered a migration blocker.
> There is no armel binary in testing.
Interesting. I did not notice this before and had simply assumed it was in testing because the version that is in testing for other platforms (2020.02.17-1) is listed as “Installed” for armel on the QA page [0]. The logs indicate its tests have never passed on armel, so I’m not sure what “Installed” is referring to here.
[0]: https://buildd.debian.org/status/package.php?p=rumur&suite=bullseye
>> What is the proper way to address such timeouts?
> That depends on their reason. Looks like the tests are indeed very
> time-consuming and/or CPU-heavy, as on mips64el they laster for an hour.
Perhaps? Time-consuming or CPU-heavy is relative, but maybe these would fall into that category. They run in 10-15 minutes on a local underpowered dev machine, but there was a prior Debian bug about this test suite hammering a build machine (#951497).
>> This package has a pretty standard cmake/ctest
>> setup, so I would also be interested in hearing from anyone who knows
>> how to prevent hiding test suite stdout/stderr on success without
>> constructing a manual ctest --verbose invocation in CMakeLists.txt.
> A "pretty standard setup" would have many small tests, while this one
> evidently only has one large "Tests" test that calls some hand-written
> Python script. I don't know anything about ctest but I guess it eats that
> script's stdout by default?
Yes, ctest swallows the normal stdout of the test script. For local dev and in CI I typically avoid this by running the test script directly, but I wanted as little magic in my debian/rules as possible so I let it do whatever its CMake defaults were.
Reply to: