[csw-maintainers] Garbage collection in allpkgs

[Maciej (Matchek) Bliziński]

2012/12/31 Dagobert Michelsen <dam at opencsw.org>:
>>>> I'm not sure what you mean by downstream-sites selecting what to
>>>> offer.
>>>
>>> Official sites mirroring our packages.
>>
>> I was asking what do you mean by selecting what to offer. Downstream
>> sites I understand.
>>
>>>> The primary mirror has a set of files, and that's it.
>>>
>>> Not quite. There are all files in the filesystem avaialable for download.
>>> However, if you rsync "opencsw" you won't get allpkgs/, so almost all
>>> of the official mirror sites don't mirror allpkgs/.
>>
>> So there's a set of file that you get when you rsync and that's it. If
>> you rsync, you don't get to choose, you get what you it's given to
>> you. If you don't, then you're not a full mirror.
>
> You are completely missing the point here. Please try
>   rsync rsync://mirror.opencsw.org/opencsw/
> and compare this to
>   rsync rsync://mirror.opencsw.org/opencsw-full/

I don't think I've ever come across the opencsw-full URL. Our
conversation could have been:

Me: "I'm not sure what you mean by downstream-sites selecting what to offer."
You: "There are 2 directories you can sync, /opencsw and
/opencsw-full, it's documented at <url>."

It would have saved us confusion.

> The former does not include allpkgs/, the latter does. By choosing the rsync URL
> you as downstream mirror decide if you want to include allpkgs or not although
> it is on the filesystem of the primary. This is rsync-magic. Trust me! :-)
>
>>>> People
>>>> can make snapshots from different points in time, is that what you
>>>> mean?
>>>
>>> No, that is different. We don't do this ATM, but archive catalog-files,
>>> so if someone has a specific problem we can regenerate everything
>>> from that catalog and allpkgs/ and this is another reason why I think
>>> having allpkgs is a Good Thing™.
>>
>> Did we ever do that? Did we even exercise doing this? I think that in
>> a real situation we would do something else rather than recreating a
>> past catalog. Do you have a specific scenario in mind? For example, we
>> have a bad, I don't know, MySQL. What would make us recreate an old
>> catalog from archives, rather than selectively solving the problem at
>> hand?
>
> I did a couple of times.

Fair enough then.

Maciej