[csw-maintainers] Catalog checks via REST

[Maciej (Matchek) Bliziński]

2012/1/15 Peter Bonivart <bonivart at opencsw.org>:
> 2012/1/15 Maciej (Matchek) Bliziński <maciej at opencsw.org>:
>> Can you rephrase? I'm not sure how you mean. There is no catalog
>> generation on the web backend (pkgdb_web.py), it's done from a cron
>> job.
>
> Yes, but if the database was complete, as in having all needed info to
> create a catalog, it would take no time to temporarily generate and
> check one integrated in the upload procedure.

The main topic is how we know that something's broken. We're lacking a
piece of code which would let us know that something's broken. Whether
it's in a web backend or a cron job, is a secondary issue. The
important one is that we receive a meaningful information when things
break. I thought that this could be done by a separate piece of code,
following the KISS principle.

The bad catalogs aren't pushed, so parsing catalog files could not be
used for this purpose, only the REST interface allows access to that
data.

As Dago noted, the way the data are provided right now is quite slow
(although I didn't see any numbers), a speedup will require database
schema change. I can't give any time line right now, if anyone cares
about improving our database backend, I'll be happy to help them
familiarize with the code base.

Maciej