[csw-maintainers] Shared library placement proposal

[Maciej Bliziński]

Hi Jonathan,

2011/2/15 Jonathan Craig <jcraig at opencsw.org>:
> 2011/2/15 Maciej Bliziński <maciej at opencsw.org>:
>> 2011/2/15 Jonathan Craig <jcraig at opencsw.org>:
>>> The only section I felt was somewhat harry was the portion that spoke
>>> about linking to an older library.  This portion pre-supposes a
>>> solution to this problem when it hasn't really been discussed (at
>>> least in the context of this policy).
>>
>> It's a good time to discuss it.
>>
>
> We can certainly discuss it but do we wish to hold up progress on
> library placement for a sidebar discussion on how to support multiple
> versions of a given runtime library for developers?  One of the goals
> was to decouple the shared library issue from how to support multiple
> versions of a given application and then the proposal drives right
> into this issue.

True, although the proposal only deals with shared libraries.  In most
cases, supporting multiple versions is about conflicting executable
names, e.g. in mysql or postgresql.  We could take this issue out of
the proposal, but it seems to me like the two issues are so closely
related, they are best covered by a single document.

>>> I think the subject of how to
>>> enable access to older libraries for end-user developers merits its
>>> own discussion and/or policy.  The method described in the document
>>> has some serious downfalls
>>
>> Could you outline them?
>
> The original example of devoid of packaging information.  I'm assuming
> at least two packages exist (libfoo1, libfoo2), but which one owns the
> /opt/csw/lib/libfoo.so -> libfoo.so.N?

The libfoo.so link belongs the libfoo_dev (CSWlibfoo-dev) package, and
there's only one symlink.  In most cases, we don't want to link to the
old library, so there's one symlink, and it points at the newest
version.  Therefore, in the typical (no need to link against the old
library), we have:

CSWlibfoo1 (lib/libfoo.so.1)
CSWlibfoo2 (lib/libfoo.so.2)
CSWlibfoo_dev (lib/libfoo.so -> libfoo.so.2)

> The example layout supposes the maintainer do a lot of monkeying about
> to shuffle libraries around and, pardon my emotional response, feels
> messy.  If the library includes its own private prefix
> (/opt/csw/{prefix}) then some stuff is shuffled to
> /opt/csw/lib/{prefix} and other stuff is kept in
> /opt/csw/{prefix}/lib.  It feels confusing and fractured, but thats
> just my opinion and I'm willing to hear counter arguments.

The fragment about linking to older libraries was meant to show an
example - how it can be done without a separate prefix.  I wanted to
counter the argument that this is something that requires compiling
the whole package in to a separate prefix.

> Could you talk through the design choices  involved with the layout
> shown in the proposal example over alternatives?

There are two options: using a separate prefix, and using subdirectories.

The --prefix way (used with berkeleydb):

CSWlibfoo1
/opt/csw/libfoo1/lib/libfoo.so.1
CSWlibfoo1_dev
/opt/csw/libfoo1/include/foo.h
/opt/csw/libfoo1/lib/libfoo.so -> libfoo.so.1

CSWlibfoo2
/opt/csw/libfoo1/lib/libfoo.so.2
CSWlibfoo2_dev
/opt/csw/libfoo2/include/foo.h
/opt/csw/libfoo2/lib/libfoo.so -> libfoo.so.2

In this scenario, you always have to specify
-I/opt/csw/libfoo1/include or -I/opt/csw/libfoo2/include, and
-L/opt/csw/libfoo1/lib or -L/opt/csw/libfoo2/lib, otherwise the
software you're compiling will not be able to find libfoo.  Even if
you want to link against the newest version (the default), you still
have to specify these options.

If you bring 64-bit libraries into the picture, the layout needs to
include the 32 and 64 links:

CSWlibfoo1
/opt/csw/libfoo1/lib/libfoo.so.1
/opt/csw/libfoo1/lib/sparcv9/libfoo.so.1
/opt/csw/libfoo1/32 -> .
/opt/csw/libfoo1/64 -> sparcv9
CSWlibfoo1_dev
/opt/csw/libfoo1/include/foo.h
/opt/csw/libfoo1/lib/libfoo.so -> libfoo.so.1
/opt/csw/libfoo1/lib/sparcv9/libfoo.so -> libfoo.so.1

CSWlibfoo2
/opt/csw/libfoo2/lib/libfoo.so.2
/opt/csw/libfoo2/lib/sparcv9/libfoo.so.2
/opt/csw/libfoo2/32 -> .
/opt/csw/libfoo2/64 -> sparcv9
CSWlibfoo2_dev
/opt/csw/libfoo2/include/foo.h
/opt/csw/libfoo2/lib/libfoo.so -> libfoo.so.2
/opt/csw/libfoo2/lib/sparcv9/libfoo.so -> libfoo.so.2


The second option, using subdirectories (and 64-bit libs):

CSWlibfoo1
/opt/csw/lib/libfoo.so.1
/opt/csw/lib/sparcv9/libfoo.so.1
CSWlibfoo1_dev
/opt/csw/include/libfoo1/foo.h
/opt/csw/lib/libfoo1/libfoo.so -> ../libfoo.so.1
/opt/csw/lib/libfoo1/sparcv9/libfoo.so -> ../../sparcv9/libfoo.so.1

CSWlibfoo2
/opt/csw/lib/libfoo.so.2
/opt/csw/lib/sparcv9/libfoo.so.2
CSWlibfoo_dev
/opt/csw/include/foo.h
/opt/csw/lib/libfoo.so -> libfoo.so.2
/opt/csw/lib/sparcv9/libfoo.so -> libfoo.so.2

In this scenario, you can pass the standard flags (-I/opt/csw/include
and -L/opt/csw/lib) and your software will link to the newest version
of libfoo.  If an older version is necessary, you need to pass
additional -I and -L.

>>> and I would prefer a method more like the
>>> one used by ubuntu.  This places the include.h and the libfoo.so link
>>> into a versioned package like libfoo2.0-dev.  This allows the end-user
>>> to choose the default development version of the library to use with
>>> builds by choosing the approriate -dev package.  The libfoo-rt package
>>> then would only have the libfoo.so.1 file needed by executables.
>>>
>>> Example:
>>> appfoo1
>>>  /opt/csw/bin/appfoo (depends on /opt/csw/libfoo.so.1)
>>>
>>> libfoo1rt
>>
>> I guess that would be libfoo1 / CSWlibfoo1.
>>
>>>  /opt/csw/lib/libfoo.so.1
>>>
>>> libfoo1-dev
>>>  /opt/csw/include/foo.h
>>>  /opt/csw/lib/libfoo.so -> libfoo.so.1
>>>
>>> appfoo2
>>>  /opt/csw/bin/appfoo (depends on /opt/csw/libfoo.so.2)
>>>
>>> libfoo2rt
>>>  /opt/csw/lib/libfoo.so.2
>>>
>>> libfoo2-dev
>>>  /opt/csw/include/foo.h
>>>  /opt/csw/lib/libfoo.so -> libfoo.so.2
>>>
>>> Now libfoo1rt and libfoo2rt can coexist and the end-user can choose
>>> which library they want to develop against by choosing the appropriate
>>> -dev package.
>>
>> In our case it can't be done by including files directly in packages.
>> If you tried to release the -dev packages like this, your second
>> package would be rejected with a note that it conflicts with the first
>> one.  Perhaps an approach using alternatives could be used: place
>> files elsewhere and provide symlinks.
>
> Cant be done given the existing implementation or cant be done at all.
>  Should we limit our solutions to the existing code if it precludes
> the implementation of a more elegant solution?

In short, can't be done at all, unfortunately.

The issue of conflicting files stems from the workings of Solaris
packages.  In dpkg for example, it is not allowed for one file to be
owned by two packages.  IIRC, you can create two packages providing
the same file, but dpkg will not allow you to install them.  In
Solaris, pkgadd will return a success exit code and will overwrite the
existing file on disk.  In this scenario, whichever package was last,
wins.  This is not acceptable, so we need to enforce file uniqueness
by never creating packages with conflicting files.

The issue surfaced not so long ago, so there still is a good deal of
conflicting files in the catalog[1].

If we encounter a case of colliding files, we need to either rename
one or both files, or use alternatives.

>> There's one issue to be addressed: we are using shared build systems
>> on the buildfarm (e.g. current9s).  It needs to be possible to link to
>> libfoo.so.2 as well as libfoo.so.1 on a single system, without any
>> global package manipulation (and root / sudo access).
>
> Hmm, this is a tough one because its a situation we have as a
> maintainer community that isn't as much of an issue for an end user.
> An end user can install/re-install the -dev package as needed.  Access
> to the shared library could be handled by creating a "lib" directory
> in the package build space that creates the symlink to whatever
> version of the shared library.  This doesn't solve the issue of
> conflicting include files.  For the build farm could we have
> installation of development packages into private root directories
> that are added to the using -L/-I options but expect to find their
> runtime libraries in /opt/csw/lib using -R.

It would mean that we release development packages that we don't use
ourselves.  We would need to have an additional testing environment.
That's a relatively large amount of work, where the group benefiting
from it is probably very small, since we rarely hear from people
building software using packages; and when we do, they usually become
maintainers.

Going back to the library placement proposal - if the linking against
old libraries section is going to grow more, it should be taken out to
a separate document.  The proposal can simply mention that it does not
prevent older shared libraries from being linked to.

What do you think?

Maciej

[1] http://buildfarm.opencsw.org/pkgdb/error-tags/file-collision/