Re: nocheck (don't run) vs nodoc (don't build)
On Wed, 26 Apr 2023 at 23:24:58 +0200, Christian Kastner wrote:
> On 2023-04-26 20:42, Russ Allbery wrote:
> > It's just
> > less common (although certainly not unheard of) for test suites to have
> > test-suite-only build dependencies (as opposed to test-only runtime
> > dependencies, which are very common in at least the Perl world).
>
> We (the ROCm team) have a few instances of such test suites, namely
> header-only libraries. Upstream ships tests that require compilation,
> and we'd like to ship these tests for autopkgtest purposes.
I think test-suite-only build dependencies are actually quite common.
At least, I see them a lot.
But DEB_BUILD_OPTIONS are not a mechanism to reduce build-dependencies:
they *cannot* reduce build dependencies. If you want to reduce
build-dependencies, you have to use build profiles.
nocheck is both a build option and a build profile, with a dependency
between them: using the build profile documented as requiring also using
the option (in other words, if you try to build with DEB_BUILD_OPTIONS=""
and DEB_BUILD_PROFILES=nocheck, that's undefined behaviour and it's OK
for packages to FTBFS). So the three valid things are:
DEB_BUILD_OPTIONS="" DEB_BUILD_PROFILES="" - normal
DEB_BUILD_OPTIONS="nocheck" DEB_BUILD_PROFILES="" - don't run
the test suite to save some time or work around known failures
DEB_BUILD_OPTIONS="nocheck" DEB_BUILD_PROFILES="nocheck" - same, but
also try to avoid installing test-only build-dependencies where feasible
Any of these can be combined with DEB_BUILD_PROFILES += " noinsttest"
to disable the build of "as-installed" tests like fwupd-tests.
> Strictly speaking, [the build-profiles spec] says build dependencies
> for the purposes of
> _running_ should be ignored, which makes perfect sense of course. But
> what if there is another purpose, specifically shipping autopkgtests?
> Like packages 'systemd-tests', or 'fwupd-tests'?
Then you probably need the same logic as in src:dbus and src:glib2.0,
which is this pseudocode, so that you'll compile with tests enabled
whenever there is at least one reason to want them:
Package: foobar
Build-Depends: thing-needed-to-compile-tests <!nocheck> <!noinsttest>,
thing-needed-to-run-tests <!nocheck>
Package: foobar-tests
Build-Profiles: <!noinsttest>
# note: OPTIONS not PROFILES
want build-time tests := not (nocheck in DEB_BUILD_OPTIONS)
# note: PROFILES not OPTIONS
want installed-tests := not (noinsttest in DEB_BUILD_PROFILES)
# or equivalently and often clearer:
# want installed-tests := (foobar-tests in dh_listpackages output)
override_dh_auto_configure:
if want build-time tests or want installed-tests:
--enable-tests or equivalent
else:
--disable-tests or equivalent
# recent dh_auto_test does this internally, but you might still need
# this logic if the tests have non-trivial setup/teardown
override_dh_auto_test:
if want build-time tests:
test setup
dh_auto_test
test teardown
else:
do nothing
I can never remember without looking it up whether
<!nocheck> <!noinsttest> or <!nocheck !noinsttest> is the right syntax
for this, but I'm reasonably sure I got it right in dbus and glib2.0 (by
testing all four combinations), so it must be <!nocheck> <!noinsttest>.
Any build-profile is OK to leave unimplemented (as long as nobody is
relying on it to avoid cyclic dependencies during bootstrapping, and
if they are, they'll tell you); so if you don't care about minimizing
build-dependencies, it's OK to simplify this by behaving as though
noinsttest was never used. But in that case you would always have to
build your tests (or at least the subset of them that end up installed
in foobar-tests), even if they will not be run.
> But that will take me a while, as I clearly need to think this through a
> bit more. Because it just dawned me that for a header-only library, the
> argument could be made that autopkgtests should always build the tests
> from scratch anyways. Or maybe even both (run, and build-and-run).
For the special case of a header-only library, yes I think the autopkgtest
should ideally rebuild the test executable every time, because that's
what real consumers of your API have to do, and it confirms that if your
header-only library is doing some tricky C++ thing, it hasn't been broken
by a newer toolchain. (But for autopkgtest, it should really be compiled
against the header-only library from /usr/include, and not against the copy
of hopefully the same headers in the source tree!)
For the more common case of a shared library, I think it's desirable to
*not* rebuild the tests every time, as a way to verify that the ABI of
the new shared library does not cause pre-existing binaries to regress.
If the test was rebuilt every time, it would silently accept and ignore
ABI breaks, by always being compiled with the new header that describes
the new ABI.
Header-only libraries are special because they don't have ABI, only API.
smcv
Reply to: