[csw-maintainers] Catalog checks via REST

[Maciej (Matchek) Bliziński]

2011/12/29 Ben Walton <bwalton at opencsw.org>:
> Excerpts from Maciej (Matchek) Bliziński's message of Tue Dec 27 13:23:28 -0500 2011:
>
> Hi Maciej,
>
>> The current situation with the catalogs is that we don't have good
>> diagnostics of catalogs. If something goes wrong, the new catalog is
>> not pushed, but we have no feedback about what was wrong, until
>> Dagobert pulls out a cron email from the depths of his email inbox.
>
> I've considered routing a few mails of this nature to devel@ or
> buildfarm@ so that they're seen by more people than me.  Specifically,
> I'm considering the mantis and web db update scripts, but others such
> as catalog generation would be good candidates too.

Yes, this mailing list would be a good target, but in general, we have
to be careful with targeting mailing lists with automated emails. We
have the code commits currently, but code commits are human-generated.
Emails from cron are different, they keep coming day in and day out
whether they hold any interesting information or not. We need to keep
the email traffic meaningful.

> If this were
> done, we'd want to move the execution of the scripts to role accounts
> (this should be done anyway) so that multiple people can jump in to
> look at issues if required.

+1

> Using the REST URLS, you'd need to rewrite the json -> catalog code
> (not huge, but still redundant).  If we simply routed the email
> differently, we'd get a heads up and not duplicate code.

Don't we have duplicate code already? There is a Ruby and a Python
implementation.

I'm not sure if ckcat can generate emails when necessary while keeping
silent when there's nothing interesting to say.

> If you're talking about new, deeper, checks on the catalog than what
> chkcat currently does then ignore the above paragraph.  I still think
> getting this type of mail in front of multiple people is a good idea
> though.

Part of the idea is to have asynchronous checks. You could detect
problems even before the catalog is generated.

Maciej