[csw-maintainers] automated catalog promotion for packages

[Peter FELECAN]

Ben Walton <bwalton at opencsw.org> writes:

> Do we agree that all package removals should be done manually or does
> anyone think this should be done automatically too?

Manual process but enabled for the maintainer, i.e., a maintainer can
remove his package. There is an exception to that: see bellow where I'm
discussing a subsequent update.

> For package adds and updates we'll need to determine the point in time
> that a package (add or update) enters the unstable catalog.  The
> simplistic view would use the REV= date stamp for adds/updates but
> that can be misleading as a package might sit in experimental for an
> unknown amount of time before being pushed.  Thus we need to watch the
> catalog(s) and detect package entry.  This is easy enough to do as we
> have machinery for this task already.  The initial spotting of a new
> package should trigger a clock for this package that upon expiry will
> see it moved forward.

Clocking must depend on the entry epoch of the package and not on the
REV attribute.

> What criteria can be used to stop the clock?  Some of the following
> are obvious and some are possibilities.  I'm sure I'm missing items
> here too.
>
> 1. A bug against the package in question.

Yes, but only for bugs of a certain level, e.g., for major and critical
ones; however, the maintainer must have the ability to change the level
of a bug if he choose so.

> 2. A bug against another package in the same 'bundle.'

Yes.

> 3. A subsequent update of the same package.

>From my standpoint, when and update enters in the unstable catalog and
there is still a previous version of that packaged, clocking or not, the
previous package is removed automatically.

> 4. A bug against a dependency if it has a ticking promotion clock too.

I don't get that because a bug for the dependency would stop its clock,
isn't it?

> All of the bug-based items are complicated by the fact that mantis
> carries no information about which catalog the particular bug is
> relevant too.  A bug could be filed against the package in the named
> release, not the version in unstable but we have no way to ascertain
> that currently.
> To handle the mantis limitations, we could either add a facility to
> the promotion tool to ignore various bug id's (a manual action) or
> look at alternate bug tracking systems.

The Mantis schemata and interface can be changed, it's a practice that
I've seen in some places; this will help to keep the current bug tracker
with minimal effort, versus changing toward an hypothetically one which
is functionally complete.

> The first item, barring the limitations, is a no brainer.  A bug stops
> the clock.  The question here is whether we remove this package from
> our (long lived) promotion object outright or mark it stalled?  If we
> remove it outright then the third action would never occur...we'd just
> see another new package later on.  If we do this, the 'override this
> bug' mechanism would need to repopulate this entry into the promotion
> object instead of just marking a bug as ignorable.  Leaving the bug in
> the object but marking it stalled has impact on subsequent spottings
> of the same package.

The ignore bug mechanism must be implemented in the bug tracker, i.e.,
it's a new attribute of the bug, and must be available only to the
maintainer.

> A subsequent update of the same package could simply overwrite the
> existing entry in the current promotion tracking object.  That would
> be the equivalent of saying "we assume that a new update addresses
> open bugs.  it would require a new bug to stop the clock again."  Does
> that seem like a valid assumption to build in?

Quite, but maybe we should add a package attribute explicating the bugs
corrected if any; many packaging systems have this very useful
information.

> I think the dependency issue will be the biggest trap here.  I think
> we need to track bugs on dependencies as a broken dep could mean a
> broken app.  We may want to only consider deps that also have ticking
> clocks.  This would allow for the fact that the package under
> consideration was built against the dep (library, etc) that is already
> in the named release catalog.  A dep that pre-exists in the promotion
> object would (assuming we keep the buildfarm very fresh) mean that the
> new package was built against the dep package that has a ticking clock
> itself.

I don't get this specific case. A dependency is a dependency and if it
has a bug, its clock is stopped, isn't it?

> We could choose to ignore dependencies, but I'm not sure that's a good
> idea.

Definitely a bad thing.

> If we decide that blocking a package based on its dependencies is the
> best action, then we have further complications.  An update of the
> dependent package must also remove the block that it set on the
> original.  I guess that's more of a bookkeeping issue in the data
> structure than a hard challenge but it's still something to consider.

It's just bookkeeping.

> We should also offer a mechanism for a maintainer to signal that a
> package is not suitable for promotion.  That could come in the form of
> a bug filed or through a command that is run.  I think filing a bug is
> more consistent (and visible) in this case.

There should be a special bug level for a better visibility of the
maintainer's intent.

> Now, if we end up with several packages in the promotion object that
> have stopped clocks, we'll want to eventually garbage collect them.
> We should be able to set quite a long expiry time on this but at some
> point, things will fall out the bottom.  Do we need a way for the
> maintainer to signal that the clock should start again?  I think the
> simplest way to achieve that is to have the maintainer submit the
> package again and start a new clock.  (This would require a different
> REV= stamp to ensure it is detected.)

No REV stamp difference needed as the clocking is not based on that but
on the catalog entry or re-entry time. Consequently, there should be a
re-entry procedure, manual, of course.

> Now, what about different architectures and os releases?  The bug
> system isn't granular enough to pick out this info so I think we need
> to stall the package in all catalogs until the normal criteria are
> met.  This would have the benefit of keeping catalogs in lock step
> which I think is a good thing.

No need for this if the bug tracking system give the possibility to
signal a bug on a specific release, architecture and package revision
(which implicates the catalog).

-- 
Peter