[csw-devel] SF.net SVN: gar:[21155] csw/mgar/pkg/opencsw-manual/trunk/files
wahwah at users.sourceforge.net
wahwah at users.sourceforge.net
Tue May 21 09:56:54 CEST 2013
Revision: 21155
http://gar.svn.sourceforge.net/gar/?rev=21155&view=rev
Author: wahwah
Date: 2013-05-21 07:56:50 +0000 (Tue, 21 May 2013)
Log Message:
-----------
opencsw-manual/trunk: catalog format + 32-bit/64-bit support
Modified Paths:
--------------
csw/mgar/pkg/opencsw-manual/trunk/files/for-developers/index.rst
csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/building-a-catalog.rst
csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/index.rst
Added Paths:
-----------
csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/32-bit-and-64-bit.rst
csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/catalog-format.rst
Modified: csw/mgar/pkg/opencsw-manual/trunk/files/for-developers/index.rst
===================================================================
--- csw/mgar/pkg/opencsw-manual/trunk/files/for-developers/index.rst 2013-05-20 21:00:15 UTC (rev 21154)
+++ csw/mgar/pkg/opencsw-manual/trunk/files/for-developers/index.rst 2013-05-21 07:56:50 UTC (rev 21155)
@@ -5,6 +5,8 @@
This is a manual for developers who want to build own software, using
tools and libraries provided by OpenCSW.
+.. _linking against OpenCSW libraries:
+
Linking against OpenCSW libraries
=================================
@@ -25,6 +27,14 @@
LDFLAGS="-L/opt/csw/lib/64 -R/opt/csw/lib/64"
PKG_CONFIG_PATH="/opt/csw/lib/64/pkgconfig"
+64-bit libraries live in the ``/opt/csw/lib/sparcv9`` and/or
+``/opt/csw/lib/amd64`` directories. The ``/opt/csw/lib/64`` path is
+a symlink to a chosen architecture subdirectory. For example, on SPARC
+``/opt/csw/lib/64`` is a symlink to ``/opt/csw/lib/sparcv9``. All
+binaries compiled with the ``-R/opt/csw/lib/64`` flag will try to look
+at that path and find their libraries.
+
+
.. _LD_LIBRARY_PATH - just say no:
https://blogs.oracle.com/rie/entry/tt_ld_library_path_tt
Added: csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/32-bit-and-64-bit.rst
===================================================================
--- csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/32-bit-and-64-bit.rst (rev 0)
+++ csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/32-bit-and-64-bit.rst 2013-05-21 07:56:50 UTC (rev 21155)
@@ -0,0 +1,150 @@
+--------------------------
+32-bit and 64-bit packages
+--------------------------
+
+64-bit binaries aren't always best
+----------------------------------
+
+It's counter-intuitive, but 64-bit binaries are more memory-hungry and
+often slower than 32-bit ones. It makes sense to build 64-bit binaries
+when:
+
+* There's a measurable speed boost.
+* The application needs to access more than 2G of RAM.
+* The application needs to open more than 256 file descriptors.
+* The application must match the kernel because it accesses its internal
+ structures. One example is the ``lsof`` utility.
+
+There's also the 2G file size restriction, but it can be worked around
+by adding the ``-D_LARGEFILE64_SOURCE=1`` flag to the C preprocessor.
+
+32-bit or 64-bit ‒ Executables
+------------------------------
+
+When building packages, the general guideline with regard to bitness is
+that executables should default to 32-bit, unless there are reasons to
+default them to 64-bit. So the default would be::
+
+ /opt/csw/bin/foo (32-bit binary)
+ /opt/csw/bin/sparcv9/foo (64-bit binary)
+
+If ``/opt/csw/bin`` is in PATH, the 32-bit binary will be run by
+default, but if the user wants, they can run the 64-bit binary as well.
+
+To run the 64-bit binary where/when possible, you can use isaexec::
+
+ /opt/csw/bin/foo → isaexec (hardlink)
+ /opt/csw/bin/sparcv8/foo (32-bit binary)
+ /opt/csw/bin/sparcv9/foo (64-bit binary)
+
+The ``isaexec`` program is a wrapper which detects the architecture and
+looks for the most advanced binary the current architecture can run. On
+sparc, it almost always means running a 64-bit binary. So if you use
+``isaexec``, it means defaulting to 64-bit in most cases.
+
+32-bit or 64-bit ‒ Shared libraries
+-----------------------------------
+
+The policy regarding shared libraries is different: if possible, do
+include 64-bit shared libraries in your package. Even though they might
+not be used by default, certain projects might significantly benefit
+from them at some point. One example would be ImageMagick, which
+performs better in 64-bit mode, and requires numerous shared libraries
+to run.
+
+.. _64-bit header files:
+
+Development packages (header files)
+-----------------------------------
+
+Development packages in most cases don't distinguish between 32-bit and
+64-bit ‒ you don't have to do anything.
+
+However, there are some software projects (e.g. ``gmp``) which install
+different headers depending on bitness. These have to be handled
+specifically, but providing both 32-bit and 64-bit header files with
+a specific switch file. For example::
+
+ /opt/csw/include/foo.h (the switch file)
+ /opt/csw/include/foo-32.h
+ /opt/csw/include/foo-64.h
+
+The contents of ``foo.h`` would be::
+
+ /* Allow 32 and 64 bit headers to coexist */
+ #if defined __amd64 || defined __x86_64 || defined __sparcv9
+ #include "foo-64.h"
+ #else
+ #include "foo-32.h"
+ #endif
+
+To implement the following in GAR, you need to set ``EXTRA_PAX_ARGS`` to
+rewrite specific header names to be bit-specific, and then manually
+install the switch file::
+
+ EXTRA_PAX_ARGS_32 = -s ",^\.$(includedir)/foo.h$$,.$(includedir)/foo-32.h,p"
+ EXTRA_PAX_ARGS_64 = -s ",^\.$(includedir)/foo.h$$,.$(includedir)/foo-64.h,p"
+ EXTRA_PAX_ARGS = $(EXTRA_PAX_ARGS_$(MEMORYMODEL))
+
+ (...)
+
+ include gar/category.mk
+
+ post-merge:
+ ginstall $(FILEDIR)/foo.h $(PKGROOT)$(includedir)/foo.h
+ @$(MAKECOOKIE)
+
+This operation must happen in the post-merge stage, because this
+operation must be done after we've built binaries for all architectures
+and we're merging them into a single directory tree. See the
+`modulations in GAR video`_ for more information.
+
+Other files which happen to be 32-bit or 64-bit specific
+--------------------------------------------------------
+
+Usually only binaries and libraries differ; all other files have the
+same content regardless to the architecture. However, some software
+projects might embed architecture specific information into files which
+aren't binaries and don't have a mechanism of choosing the right version
+of the file at runtime::
+
+ /opt/csw/bin/foo → isaexec (hardlink)
+ /opt/csw/sparcv8/foo (32-bit binary, has /opt/csw/lib in RUNPATH)
+ /opt/csw/sparcv9/foo (64-bit binary, has /opt/csw/lib/64 in RUNPATH)
+ /opt/csw/lib/64 → sparcv9 (symlink)
+ /opt/csw/lib/libfoo.so.1 (32-bit library)
+ /opt/csw/lib/sparcv9/libfoo.so.1 (64-bit library)
+ /opt/csw/lib/foo/arch_specific_data (ZONK! no mechanism to differentiate!)
+
+The `64-bit header files`_ example shown above is a lucky case, because
+header files follow the C syntax, so you can use the ``#if defined``
+conditional statement. In the general case there is no single solution,
+it all depends on whether you can write a conditional statement such
+that you get the 32-bit or 64-bit content depending on a choice made at
+runtime.
+
+
+Compiling 64-bit binaries
+-------------------------
+
+To compile a 64-bit binary, you add ``-m64`` to the compiler invocation.
+See more about :ref:`linking against OpenCSW libraries`.
+
+In GAR, there's a shortcut::
+
+ BUILD64 = 1
+
+
+**See also**
+
+* `Solaris 64-bit Developer's Guide`_
+* `Are 64-bit Binaries Really Slower than 32-bit Binaries?`_
+
+.. _Solaris 64-bit Developer's Guide:
+ http://docs.sun.com/app/docs/doc/816-5138
+
+.. _Are 64-bit Binaries Really Slower than 32-bit Binaries?:
+ http://www.osnews.com/story/5768
+
+.. _modulations in GAR video:
+ http://youtu.be/7I3efByIg84
Modified: csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/building-a-catalog.rst
===================================================================
--- csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/building-a-catalog.rst 2013-05-20 21:00:15 UTC (rev 21154)
+++ csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/building-a-catalog.rst 2013-05-21 07:56:50 UTC (rev 21155)
@@ -2,24 +2,91 @@
Building a package catalog
--------------------------
-If you have built a package (or a set of them), you might want to test them.
-You could just ``gunzip`` the packages and run ``pkgadd``, but if you wanted to test
-your package on a couple of systems, it's better to build a catalog.
+If you have built a package (or a set of them), you might want to test
+them. You could just ``gunzip`` the packages and run ``pkgadd``, but if
+you wanted to test your package on a couple of systems, it's better to
+build a catalog.
Make sure you've installed the ``pkgutilplus`` package. It contains the
-``bldcat`` utility. You can use it to create a local catalog with your
-package::
+``bldcat`` utility. You can use it to create a local catalog containing
+your package::
catalog_root=${HOME}/opencsw-catalog
catalog_path=${catalog_root}/$(uname -p)/$(uname -r)
cp /path/to/your_package.pkg ${catalog_path}
bldcat ${catalog_path}
-Once this is done, you can instruct ``pkgutil`` to install packages from it. You
-can either serve the root catalog directory over HTTP, or you can refer to
-a local filesystem path::
+Once this is done, you can instruct ``pkgutil`` to install packages from
+it. You can either serve the root catalog directory over HTTP, or you
+can refer to a local filesystem path::
sudo pkgutil -t file://${catalog_path} -y -i your_package
If you create a persistent local catalog, you can add the path or URL to
-``/opt/csw/etc/pkgutil.conf``.
+``/etc/opt/csw/pkgutil.conf``.
+
+Building your own catalogs
+--------------------------
+
+If you have test packages that you have built for OpenCSW or your own
+packages that have dependencies on each other, you may want to use
+pkgutil to install them. For that to work you need a catalog for your
+packages just like the ones OpenCSW publishes.
+
+There's a simple perl script to parse your packages in a directory and
+build a catalog for them. The one argument is the directory to parse for
+``*.pkg.gz`` files and stdout is used so use normal redirection for the
+catalog::
+
+ $ bldcat .
+ xv 3.10a,REV=2008.10.15 CSWxv xv-3.10a,REV=2008.10.15-SunOS5.8-sparc-CSW.pkg.gz d887c7aed0e849471467e4944d14c2eb 1877768 CSWcommon|CSWtiff|CSWpng|CSWjpeg|CSWzlib none
+ $ bldcat . > catalog
+ $ ls -l
+ -rw-r--r-- 1 bonivart csw 167 Oct 22 17:19 catalog
+ -rw-r--r-- 1 bonivart csw 1877768 Oct 22 17:19 xv-3.10a,REV=2008.10.15-SunOS5.8-sparc-CSW.pkg.gz
+ $ more catalog
+ xv 3.10a,REV=2008.10.15 CSWxv xv-3.10a,REV=2008.10.15-SunOS5.8-sparc-CSW.pkg.gz d887c7aed0e849471467e4944d14c2eb 1877768 CSWcommon|CSWtiff|CSWpng|CSWjpeg|CSWzlib none
+
+The script is a part of the pkgutilplus (CSWpkgutilplus) package.
+
+Checking the catalog
+--------------------
+
+There's also a simple check that the catalog is correct. It checks that
+every line is eight fields and that the dependency and category fields
+begin and end with chars. It also warns if packages are not compressed
+(normal for gzip and pkg-get). It also checks for duplicates.
+
+Test run on a manipulated catalog::
+
+ # chkcat /var/opt/csw/pkgutil/catalog
+ Skipping signature.
+ Skipping comment.
+ Skipping comment.
+ Skipping comment.
+
+ ERROR! 7 fields instead of normal 8.
+ 2.2.4,REV=2008.10.01 CSWlibtoolrt libtool_rt-2.2.4,REV=2008.10.01-SunOS5.8-sparc-CSW.pkg.gz 72ae2f64521df6e18b7d665bbf11e984 82427 CSWisaexec|CSWcommon none
+
+ ERROR! 7 fields instead of normal 8.
+ rubydoc 1.8.7,REV=2008.09.19_p72 CSWrubydoc rubydoc-1.8.7,REV=2008.09.19_p72-SunOS5.8-i386-CSW.pkg.gz d47700240d7c675e5f843b03a937c28e 3032323 none
+ WARNING! Package CSWrubytk is not compressed.
+ rubytk 1.8.7,REV=2008.09.19_p72 CSWrubytk rubytk-1.8.7,REV=2008.09.19_p72-SunOS5.8-i386-CSW.pkg 2215ac92175922c593245ef577e92fc9 317259 CSWruby|CSWtcl|CSWtk|CSWcommon none
+
+
+ WARNING! Package CSWrubytk is not compressed.
+ rubytk 1.8.7,REV=2008.09.19_p72 CSWrubytk rubytk-1.8.7,REV=2008.09.19_p72-SunOS5.8-i386-CSW.pkg 2215ac92175922c593245ef577e92fc9 317259 CSWruby|CSWtcl|CSWtk|CSWcommon none
+
+ ERROR! The category field of package CSWspamassassin begins with a non-char.
+ spamassassin 3.2.5,REV=2008.10.21 CSWspamassassin
+ spamassassin-3.2.5,REV=2008.10.21-SunOS5.8-i386-CSW.pkg.gz
+ e5bd858be4a67023b02ee1e5e760295b 896877
+ CSWosslrt|CSWperl|CSWpmarchivetar|CSWpmdbi|CSWpmdigestsha1|CSWpmiosocketinet6|CSWpmiosocketssl|CSWpmiozlib|CSWpmipcountry|CSWpmldap|CSWpmlibwww|CSWpmmaildkim|CSWpmmailspf|CSWpmmailtools|CSWpmmimebase64|CSWpmnetdns|CSWpmuri|CSWpmhtmlparser|CSWzlib|CSWcommon
+ +none
+
+ ERROR! The dependency field of package CSWxv ends with a pipe char.
+ xv 3.10a,REV=2008.10.17 CSWxv xv-3.10a,REV=2008.10.17-SunOS5.8-i386-CSW.pkg.gz 9de3c40048fc8c9f79616ee388fc98f1 1731846 CSWcommon|CSWtiff|CSWpng|CSWjpeg|CSWzlib| none
+
+ Skipping signature. Exiting.
+
+The script is a part of the pkgutilplus (CSWpkgutilplus) package.
Added: csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/catalog-format.rst
===================================================================
--- csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/catalog-format.rst (rev 0)
+++ csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/catalog-format.rst 2013-05-21 07:56:50 UTC (rev 21155)
@@ -0,0 +1,29 @@
+--------------
+Catalog format
+--------------
+
+Catalog format in short::
+
+ common version package file md5 size dependencies category i-dependencies
+
+For example::
+
+ bind 9.4.2,REV=2008.07.09_rev=p1 CSWbind bind-9.4.2,REV=2008.07.09_rev=p1-SunOS5.8-sparc-CSW.pkg.gz f68df57fcf54bfd37304b79d6f7eeacc 2954112 CSWcommon|CSWosslrt net none
+
+Each field is space separated, the dependencies field can be split with
+the pipe char, like in the example above with two dependencies. The same
+goes for the category and incompatible dependencies fields.
+
+A package can only occur once in the catalog, meaning that both package
+names and catalog names must be unique in a catalog.
+
+The catalog may have to be extended to support more features like if
+there's a source package available. In that case extra fields should be
+added to the end so not to break existing tools.
+
+See also:
+
+* `Building a catalog`_
+
+.. _Building a catalog:
+ building-a-catalog.html
Modified: csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/index.rst
===================================================================
--- csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/index.rst 2013-05-20 21:00:15 UTC (rev 21154)
+++ csw/mgar/pkg/opencsw-manual/trunk/files/for-maintainers/index.rst 2013-05-21 07:56:50 UTC (rev 21155)
@@ -11,7 +11,9 @@
about-documentation
contributing-to-recipes
filesystem-layout
+ catalog-format
building-a-catalog
shared-libraries
package-naming
buildfarm-setup
+ 32-bit-and-64-bit
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
More information about the devel
mailing list