Intention to write Deqalo for Debian ISO testing reports
Dear QA Team
In Debian, before the release of an ISO image, we give it one final
round of smoke testing. This includes a bunch of tests and spot checks
to ensure that the releases are usable as intended.
Unfortunately, our platform for managing this is rather tedious, and may
discourage many people who might otherwise be involved in testing from
contributing (and lack of testing has caused big trouble at least for
the live images, see:
https://lists.debian.org/debian-live/2017/06/msg00064.html). Currently,
it's tracked in a wiki page:
https://wiki.debian.org/Teams/DebianCD/ReleaseTesting/Stretch
While it does work, working with large tables in wikis isn't everyone's
idea of fun, and it takes some extra co-ordination to deal with wiki
locks and edit conflicts. On top of that, defining additional tests,
linking them to images, and tracking all the information that's
submitted better will just make that wiki page even more monstrous.
When I used to do testing on Ubuntu ISO images, we used the Ubuntu ISO
QA tracker (http://iso.qa.ubuntu.com). It worked quite well, you can
define milestones (like releases, alphas and daily builds), products
(ISOs, VMs, etc) and various test definitions that you could link to to
those products. Multiple people can test an iso and easily report the
results, and once the person responsible for signing off an ISO is
satisfied with it, they can mark it as ready. The code for the Ubuntu
ISO QA tracker is free software, and I briefly considered suggesting
implementing it for Debian, but it's not that actively maintained, and I
don't want to be responsible for deploying more PHP code and wouldn't
really want to maintain it either.
Recently I've been toying with the idea of re-writing a minimal viable
product version of the QA tracker, that at least matches our
functionality of using the wiki and solving its biggest issues,
although, I'm allergic to committing to write anything because I know I
have a tendency to lose focus and move on to the next exciting thing.
Today I joined a commitment on twitter to code for an hour a day
(https://twitter.com/highvoltage/status/965100119505997825), and looked
over my wishlist for something that would fit nicely into ~100 hours and
this looks like a good fit.
I intend to call it Deqalo (it comes from Debian QA Logger, but it
doesn't have to be specific to Debian or just for Debian), and the plan
is to write it as a generic python module that can be used in anything,
with a web front-end written in Flask. I also intend to add an API so
that users who test can log their results from a local command line
without having to visit the web page (might also be useful for bots that
do testing). I'm also planning to use the SB-Admin theme for the web
front-end: https://blackrockdigital.github.io/startbootstrap-sb-admin/
At this stage it's mostly vapourware, today I mostly looked at the best
way to package SB-Admin (and check what we can do to update our version
of Bootstrap that's a bit outdated in the archives), so any
feedback/changes are welcome. If it's straight-out a bad idea or if
there's already something else we should be using, I'd rather hear about
it now than invest too much time into it.
So, any thoughts or suggestions are welcome.
-Jonathan
Reply to: