On Wed, Aug 31, 2022 at 02:12:49PM +0100, Colin Watson wrote:
> Package: apt
> Version: 2.5.2
> Severity: normal
>
> apt contains an arbitrary hardcoded limit on the size of tagfiles:
>
> bool pkgTagFile::Resize()
> {
> // fail is the buffer grows too big
> if(d->Size > 1024*1024+1)
> return false;
>
> return Resize(d->Size * 2);
> }
It's a limit on a single tagfile section, not the entire file. It
would be nice to have better error reporting there, arguably.
>
> I wrote the following quick test script before finding this limit:
>
> $ cat t.py
> #! /usr/bin/python3
>
> from argparse import ArgumentParser
> import tempfile
>
> import apt_pkg
>
> apt_pkg.init()
>
> parser = ArgumentParser()
> parser.add_argument("length", type=int)
> args = parser.parse_args()
>
> with tempfile.TemporaryFile() as f:
> f.write(b"Format: 1.8\nChanges:\n ")
> f.write(b"x" * args.length)
> f.write(b"\n")
> f.seek(0)
> list(apt_pkg.TagFile(f, bytes=True))
> $ ./t.py 1048677
> $ ./t.py 1048678
> Traceback (most recent call last):
> File "/home/cjwatson/./t.py", line 19, in <module>
> list(apt_pkg.TagFile(f, bytes=True))
> apt_pkg.Error: E:Unable to parse package file (1)
>
> (I'm not sure exactly why the threshold is 1024*1024+102; presumably the
> resize steps don't quite take us through exact powers of two.)
>
> We actually encountered this in practice. Somebody uploaded a kernel
> package to Launchpad with a ~2.4 MiB .changes file, mostly consisting of
> a probably-autogenerated changelog with lots of commit messages, and the
> upload failed due to this.
>
> I think I'd prefer this *not* to be configurable, to minimize situations
> where tag files can be parsed in some environments but not others. I
> don't know whether it's possible to reasonably avoid having an arbitrary
> limit at all. Even if not, 1 MiB seems pretty small compared to memory
> sizes these days; perhaps this could be raised?
One caveat is that python-apt copies that buffer when you do
list(TagFile)) or similar, as it copies the TagSection if there is
more than one reference to it and it needs to step to the next one
(it's doing lazy copy-on-write basically).
There needs to be some limit on parsing for security reasons (to prevent
DoS by starving a parser with like infinite gzip compressed tagfile), but
I'm not sure what's reasonable.
I'm fine bumping that up and adding a nicer error so we know what's
going on. The Release file, which is a single section after all, is
limited to 10 * 1000 * 1000 or what's configured as
Acquire::MaxReleaseFileSize.
I think the defaults should match, but I think this probably
ought to be configurable so you can have no limit on trusted
inputs (or with controlled runtime timeout). Not documented,
or exposed in the function call, but via a configuration
option.
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
Attachment:
signature.asc
Description: PGP signature