[csw-maintainers] Where does "category" come from?

[Dagobert Michelsen]

Hi Maciej,

Am 26.01.2012 um 18:35 schrieb Maciej (Matchek) Bliziński:
> 2012/1/26 Dagobert Michelsen <dam at opencsw.org>:
>> I noticed the catalog contains "category" as the last field like this:
>> 
>> xtideh 07.12.28,REV=2008.02.05 CSWxtideh xtideh-07.12.28,REV=2008.02.05-SunOS5.8-all-CSW.pkg.gz 0aa9756ab43d8219b8c0c54a21e5a7dc 741164 CSWcommon eng|fun|lib
>> xtidew 1.0,REV=2008.02.05 CSWxtidew xtidew-1.0,REV=2008.02.05-SunOS5.8-all-CSW.pkg.gz e37e60cbd79c8117a8a1e475f4cd5bb0 41699951 CSWcommon eng|fun|lib
>> 
>> I figure that it must be in CatalogDetail:
>> 
>> ./lib/web/pkgdb_web.py:  r'/catalogs/([\w-]+)-(sparc|i386)-(SunOS[^-]+)/', 'CatalogDetail',
>> ./lib/web/pkgdb_web.py:class CatalogDetail(object):
>> ./lib/web/pkgdb_web.py:    return render.CatalogDetail(cat_name, pkgs)
>> 
>> However, I can't find "render", just "render_response". So, where is that field stored in the database?
> 
> Catalogs files (indexes) are not generated from the database, they are
> generated straight from the packages, using bldcat, that's where the
> answer must be.

I see. It is generated from CSW_CATEGORY from pkginfo within the package:
  http://sourceforge.net/apps/trac/pkgutil/browser/trunk/bldcat#L238

However, I can't remember what is was used for. The discussions were from 2007 and
April 2008 before OpenCSW and are hence not in the mail archive. Maybe someone
can shed some light into this, probably Peter as he implemented it :-)

I have now tested a script which generates a full catalog (with the exception
of the category of course) completely from the database via REST and got the
following runtimes (with bundle :-)
  ./getcatalog  3868.28s user 5.09s system 76% cpu 1:24:26.68 total

The URLs used were
  http://buildfarm.opencsw.org/pkgdb/rest/catalogs/unstable/sparc/SunOS5.10/
  http://buildfarm.opencsw.org/pkgdb/rest/srv4/${md5}/pkg-stats/
It seems to me that the JSON encoding/decoding takes a lot of time.


Best regards

  -- Dago

-- 
"You don't become great by trying to be great, you become great by wanting to do something,
and then doing it so hard that you become great in the process." - xkcd #896