[csw-maintainers] TeXLive packaging take almost 3 days

[Dagobert Michelsen]

Hi Peter,

Am 19.11.2012 um 15:44 schrieb pfelecan at opencsw.org:

> I'm struggling to replace teTeX with TeXLive since July... Quite a big
> endeavor. Here are some statistics:
> 
> 112180 files in the installation directory
> 
> 87 packages
> 
> I'm using the split as defined by Debian with some adaptations.
> 
> The time taken for each package is summarized in the following table:
> 
> |-------+-------------------------------|
> | hh:mm | what                          |
> |-------+-------------------------------|
> | 67:09 | total_time                    |
> |-------+-------------------------------|
> | 04:17 | build_time                    |
> | 00:20 | global_prototype              |
> | 00:36 | libkpathsea6                  |
> | 00:36 | libptexenc1                   |
> | 00:30 | tetex                         |
> | 00:41 | texlive-base                  |
> | 00:44 | texlive-bibtex-extra          |
> | 00:43 | texlive-binaries              |
> | 07:02 | texlive-common                |
> | 02:48 | texlive-dev                   |
> | 00:31 | texlive-doc-base              |
> | 00:32 | texlive-doc-bg                |
> | 00:31 | texlive-doc-cs+sk             |
> | 00:32 | texlive-doc-de                |
> | 00:36 | texlive-doc-en                |
> | 00:33 | texlive-doc-es                |
> | 00:31 | texlive-doc-fi                |
> | 00:32 | texlive-doc-fr                |
> | 00:31 | texlive-doc-it                |
> | 00:31 | texlive-doc-ja                |
> | 00:31 | texlive-doc-ko                |
> | 00:31 | texlive-doc-mn                |
> | 00:32 | texlive-doc-nl                |
> | 00:31 | texlive-doc-pl                |
> | 00:32 | texlive-doc-pt                |
> | 00:31 | texlive-doc-ru                |
> | 00:31 | texlive-doc-si                |
> | 00:31 | texlive-doc-th                |
> | 00:31 | texlive-doc-tr                |
> | 00:31 | texlive-doc-uk                |
> | 00:33 | texlive-doc-vi                |
> | 00:32 | texlive-doc-zh                |
> | 00:36 | texlive-extra-utils           |
> | 00:35 | texlive-font-utils            |
> | 01:33 | texlive-fonts-extra           |
> | 00:46 | texlive-fonts-extra-doc       |
> | 00:52 | texlive-fonts-recommended     |
> | 01:33 | texlive-fonts-recommended-doc |
> | 00:33 | texlive-formats-extra         |
> | 00:37 | texlive-games                 |
> | 00:36 | texlive-generic-extra         |
> | 00:32 | texlive-generic-recommended   |
> | 00:35 | texlive-humanities            |
> | 00:34 | texlive-humanities-doc        |
> | 00:33 | texlive-lang-african          |
> | 00:33 | texlive-lang-arabic           |
> | 00:30 | texlive-lang-croatian         |
> | 00:36 | texlive-lang-cyrillic         |
> | 00:33 | texlive-lang-czechslovak      |
> | 00:41 | texlive-lang-finnish          |
> | 00:33 | texlive-lang-french           |
> | 00:32 | texlive-lang-german           |
> | 00:41 | texlive-lang-greek            |
> | 00:33 | texlive-lang-hebrew           |
> | 00:31 | texlive-lang-hungarian        |
> | 00:34 | texlive-lang-indic            |
> | 00:32 | texlive-lang-italian          |
> | 00:31 | texlive-lang-lithuanian       |
> | 00:32 | texlive-lang-mongolian        |
> | 00:36 | texlive-lang-polish           |
> | 00:31 | texlive-lang-portuguese       |
> | 00:31 | texlive-lang-spanish          |
> | 00:31 | texlive-lang-swedish          |
> | 00:32 | texlive-lang-tibetan          |
> | 00:35 | texlive-lang-vietnamese       |
> | 00:36 | texlive-latex-base            |
> | 00:34 | texlive-latex-base-doc        |
> | 02:54 | texlive-latex-extra           |
> | 01:47 | texlive-latex-extra-doc       |
> | 00:51 | texlive-latex-recommended     |
> | 00:40 | texlive-latex-recommended-doc |
> | 00:33 | texlive-latex3                |
> | 00:33 | texlive-luatex                |
> | 00:42 | texlive-math-extra            |
> | 00:33 | texlive-metapost              |
> | 00:32 | texlive-metapost-doc          |
> | 00:35 | texlive-music                 |
> | 00:35 | texlive-omega                 |
> | 00:38 | texlive-pictures              |
> | 00:36 | texlive-pictures-doc          |
> | 00:35 | texlive-plain-extra           |
> | 00:57 | texlive-pstricks              |
> | 00:40 | texlive-pstricks-doc          |
> | 00:43 | texlive-publishers            |
> | 00:40 | texlive-publishers-doc        |
> | 00:38 | texlive-science               |
> | 00:36 | texlive-science-doc           |
> | 00:32 | texlive-xetex                 |
> | 00:32 | texlive-xindy                 |
> |-------+-------------------------------|
> 
> Most of the time is spent in cswproto and patfilter utilities.
> 
> For example: for libkpathsea6 the usage is the following:
> 
>  cswproto:
> 
> 	 real 1223.12
> 	 user 1092.55
> 	 sys 139.96
> 
>         of which 1077 by repeated calls to pkgproto
> 
> pathfilter:
> 
> 	 real 2183.69
> 	 user 2182.79
> 	 sys 0.72
> 
> IMHO, there is place for some optimization.
> 
> If there is a kind soul to help on this he'll have my gratitude.

The numbers are… impressive. I have never optimized pathfilter as it usually
runs a second for average packages, so it probably leaves much room. Please use
  DEBUG_PACKAGING=1 mgar repackage
to get the actual pathfilter invocation and grab the file it is applied to so I can
rerun and profile it. I can then happily try to speed it up.


Best regards

  -- Dago


-- 
"You don't become great by trying to be great, you become great by wanting to do something,
and then doing it so hard that you become great in the process." - xkcd #896