[csw-maintainers] Preliminary package reviews

[Maciej (Matchek) Blizinski]

No dia 26 de Agosto de 2010 09:15, Dagobert Michelsen
<dam at opencsw.org> escreveu:
> Hi Maciej,
>
> Am 26.08.2010 um 08:23 schrieb Maciej (Matchek) Blizinski:
>> Not immediately, the only requirement is to run checkpkg prior to
>> running pkgdb.  The reason is that pkgdb lacks at the moment the
>> ability to unpack files and relies on the data already being there in
>> the database.
>
> But as checkpkg locks the database I guess the packages must be
> tasted sequentially, right?

SQLite supports parallel reads from a database, locking occurs only
you're writing to the database.  I'm not sure what you mean by
sequential testing -- there are some operations that must be
sequential, but there are many places where parallel processing can be
used.

>>> This makes parallel execution from user "web" produce corrupt
>>> output, right?
>>
>> Once the data is in the database, there are no restrictions on
>> parallel runs.  pkgdb only reads the data, no potential for corruption
>> there.  I could imagine some form of race condition if one process was
>> inserting thedata into the database, and the other would try to read
>> not-yet-complete records, I haven't armored the system against it.  Do
>> I need to?
>
> I don't think so. My current usecases look like this:
> - Run checkpkg against each set of packages in subdirs from experimental
> - run pkgdb against each checkpkg run to produce html for each experimental
>  catalog. This is done every 3 hours on demand ATM.
> - next: run checkpkg against each catalog on the mirror (stable, testing,
>  unstable, experimental on mirror/opencsw-future)
>    http://mirror.opencsw.org/opencsw-future/
>  I expect these do be done daily.

You can speed up the catalog runs by saving on disk I/O and md5
calculation time by providing checkpkg with md5 sums.  Here's how I'm
running checkpkg against the whole catalog:

d=$(date +%Y-%m-%d)
c=/export/mirror/opencsw/unstable/sparc/5.9
time bin/checkpkg -s \
  -M ${c}/catalog \
  -d \
  -o ${HOME}/current-${d}-error-tags.txt \
  ${c}/*pkg*

The -M flag tells checkpkg to use the information from the file named
'catalog' instead of reading the contents of the files and calculating
the md5 sums.  The same can be done for the experimental catalogs.