[csw-maintainers] automated catalog promotion for packages

[Ben Walton]

Excerpts from Peter FELECAN's message of Fri Nov 11 08:25:44 -0500 2011:

> > Do we agree that all package removals should be done manually or does
> > anyone think this should be done automatically too?
> 
> Manual process but enabled for the maintainer, i.e., a maintainer
> can remove his package. There is an exception to that: see bellow
> where I'm discussing a subsequent update.

With the exception of stub packages that we may want to auto-expire
somehow (the point Maciej and Dago discuss), I think this is a good
way to handle removals.

> Clocking must depend on the entry epoch of the package and not on
> the REV attribute.

Agreed.  You point out below though that no REV= update would be
required, but that's not easy to avoid if we're only looking at
catalog data.  We would stop the clock on a filed bug (meeting defined
criteria) but in order to restart it for this package, we'd need to
see it re-enter the catalog.  As we'd never see it actually leave the
unstable catalog, something detectable must change for it to be
noticed again.  The REV= stamp of a re-rolled package is the low
hanging fruit here.  The only other field likely to change is the
md5sum, or possibly the version field if there is a subsequent update.

> > What criteria can be used to stop the clock?  Some of the following
> > are obvious and some are possibilities.  I'm sure I'm missing items
> > here too.
> >
> > 1. A bug against the package in question.
> 
> Yes, but only for bugs of a certain level, e.g., for major and
> critical ones; however, the maintainer must have the ability to
> change the level of a bug if he choose so.

This is a good point.  I hadn't considered it last night but I agree
fully.

> > 2. A bug against another package in the same 'bundle.'
> 
> Yes.
> 
> > 3. A subsequent update of the same package.
> 
> From my standpoint, when and update enters in the unstable catalog
> and there is still a previous version of that packaged, clocking or
> not, the previous package is removed automatically.

And the clock is reset as if the old one was never considered.  I
think this goes without saying, I'm just being explicit.

> > 4. A bug against a dependency if it has a ticking promotion clock too.
> 
> I don't get that because a bug for the dependency would stop its
> clock, isn't it?

If I build against libxml2 and my new program passes the tests, a new
bug against libxml2 shouldn't hold me up.  But if libxml2 is also new
(defined by ticking clock here, but that may not be ideal?), and a bug
is filed against it, packages built against it should (?) be stopped
as well.  The idea here is that we don't want a situation where I need
a new SONAME to exist in the next catalog and that doesn't exist
because the package providing that was held up.

To truly prevent this, we may need to really leverage the pkgdb
information.

> The Mantis schemata and interface can be changed, it's a practice
> that I've seen in some places; this will help to keep the current
> bug tracker with minimal effort, versus changing toward an
> hypothetically one which is functionally complete.

Replacing the bug tracker is a big project so avoiding it (for now) is
still a good approach, imo.  If we can bend it to our needs, we
should.

Aside from augmenting categories/types of bugs, we could also consider
merging all of the packages into a single item based on their bundle
tag.  The problem here is that this is relatively new so not all
packages have it.

> > The first item, barring the limitations, is a no brainer.  A bug stops
> > the clock.  The question here is whether we remove this package from
> > our (long lived) promotion object outright or mark it stalled?  If we
> > remove it outright then the third action would never occur...we'd just
> > see another new package later on.  If we do this, the 'override this
> > bug' mechanism would need to repopulate this entry into the promotion
> > object instead of just marking a bug as ignorable.  Leaving the bug in
> > the object but marking it stalled has impact on subsequent spottings
> > of the same package.
> 
> The ignore bug mechanism must be implemented in the bug tracker, i.e.,
> it's a new attribute of the bug, and must be available only to the
> maintainer.

This is a great idea.  It's should fall out of the rest of the bug
priority handling quite nicely.  It's also makes the package control
flow consistent.  File a bug to halt, close/alter a bug to restart.
Clean.

> > A subsequent update of the same package could simply overwrite the
> > existing entry in the current promotion tracking object.  That would
> > be the equivalent of saying "we assume that a new update addresses
> > open bugs.  it would require a new bug to stop the clock again."  Does
> > that seem like a valid assumption to build in?
> 
> Quite, but maybe we should add a package attribute explicating the
> bugs corrected if any; many packaging systems have this very useful
> information.

I think Debian does something like this.  It's heading in the
direction of proper change logs  (a required Changelog.CSW in
$docdir?).  Do you see this as needing to be visible to the promotion
system?

> > We should also offer a mechanism for a maintainer to signal that a
> > package is not suitable for promotion.  That could come in the
> > form of a bug filed or through a command that is run.  I think
> > filing a bug is more consistent (and visible) in this case.
> 
> There should be a special bug level for a better visibility of the
> maintainer's intent.

A simple additional 'severity' status of "no promotion" would do the
trick nicely.  Again, this is a nice clean way to interface with the
system.

> > Now, if we end up with several packages in the promotion object
> > that have stopped clocks, we'll want to eventually garbage collect
> > them.  We should be able to set quite a long expiry time on this
> > but at some point, things will fall out the bottom.  Do we need a
> > way for the maintainer to signal that the clock should start
> > again?  I think the simplest way to achieve that is to have the
> > maintainer submit the package again and start a new clock.  (This
> > would require a different REV= stamp to ensure it is detected.)
> 
> No REV stamp difference needed as the clocking is not based on that
> but on the catalog entry or re-entry time. Consequently, there
> should be a re-entry procedure, manual, of course.

As I mentioned above, we do need some mechanism to detect the
difference.  That can be either a drop (where we allow for the cron
interval to ensure this is seen by the back end) or a re-add with a
new version/md5sum.  The former is more fragile, imo.

> No need for this if the bug tracking system give the possibility to
> signal a bug on a specific release, architecture and package
> revision (which implicates the catalog).

We don't have the currently.  Can it be added easily Sebastian?  I'm
not overly familiar with the capabilities of mantis.

Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302