[csw-maintainers] Garbage collection in allpkgs

[Dagobert Michelsen]

Hi,

Am 29.12.2012 um 15:19 schrieb Peter Bonivart <bonivart at opencsw.org>:

> On Sat, Dec 29, 2012 at 2:54 PM, Maciej (Matchek) Bliziński
> <maciej at opencsw.org> wrote:
>> I ran garbage collection in our official catalog. This means that I
>> removed files that were in allpkgs, but were not used/referenced by
>> any of our catalogs. I managed to remove about 24GB worth of unused
>> packages. The package files are not deleted, they are only moved out
>> of allpkgs.
>> 
>> Does anyone think we should keep old files forever, that is, keep more
>> than just what's in our history of releases?

We should keep packages forever in allpkgs. I suggest putting them back.

> What catalogs did you match against, current ones or also archived ones?
> 
> I think many users have looked for that lost package from back in the
> day, we had one single mirror in Germany that didn't rsync with
> --delete so they basically archived everything but I don't think it
> was official and they could stop doing that anytime.

Correct. If archived packages are offered and users consider it useful
it should be us to offer that.

> Could we do the
> same somewhere on the buildfarm but not on the master mirror? Then we
> would have an official archive for those that need it but since it
> wouldn't be used that much it would be unnecessary to mirror it, we
> would just link to it from our mirror page.

This is already the case: allpkgs/ is not included in the main rsync
offering, just in opencsw-full:

> dam at login [login]:/home/dam > rsync rsync://mirror.opencsw.org
> csw             Legacy name, please switch to the identical 'opencsw'
> opencsw         CSW Primary Mirror, use this if you are mirroring OpenCSW (the archive "allpkgs" is now in 'opencsw-full')
> opencsw-full    CSW Primary Mirror, contains full archive of old packages
> opencsw-future  The proposed future layout of the OpenCSW Primary, layout may change without notice at any time


This is done by using the exclude-directive in rsync.conf for "csw" and "opencsw":
        exclude = allpkgs HEADER.txt

Having all packages on the primary mirror is also good IMHO. This way
each downstream-site can easily select what to offer.


Best regards

  -- Dago

-- 
"You don't become great by trying to be great, you become great by wanting to do something,
and then doing it so hard that you become great in the process." - xkcd #896