[csw-maintainers] Populating private checkpkg databases via REST

[Maciej Bliziński]

On Wed, Jan 30, 2013 at 07:13:25PM +0100, Peter FELECAN wrote:
> "Maciej (Matchek) Bliziński" <maciej at opencsw.org> writes:
> 
> > While I'm wresting with the memory leak in catalog indexing, I got the
> > idea that instead of unpacking and indexing everything locally, we
> > could make people's private build hosts download data from our
> > buildfarm. What do you think?
> 
> Your proposition has merit but I think that we should keep things
> reproducible even without OpenCSW's infrastructure; in an extreme case,
> somebody wishing to build in an unconnected bunker (viz. a bank or stuff
> like that) can do it.

Sure. I wouldn't think about this data load as of a dependency, but just
as an optional speedup.

For the last month and a half or so, instead of being productive and
playing Call of Duty, I've been rewriting parts of pkgdb and part of
that was re-indexing all out packages. I didn't make it run in one
smooth go ‒ I will do that after I've finished the main set of changes
‒ because I had to deal with some code issues along the way, but I can
tell it took something like 30h to index one catalog (e.g. unstable or
dublin).  So if you could instead just download a SQL dump file and load
it, it would be much quicker.

When I'm done, I'm actually thinking of not re-running the indexing on
the buildfarm, but instead taking the data from my virtual machine and
loading them onto the buildfarm database.

Maciej
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.opencsw.org/pipermail/maintainers/attachments/20130520/d969b79a/attachment.asc>