[csw-maintainers] Python 2.7

[Maciej Bliziński]

On Thu, Jul 25, 2013 at 07:37:01PM +0200, Peter FELECAN wrote:
> "Maciej (Matchek) Bliziński" <maciej at opencsw.org> writes:
> >
> > They don't provide sources, so I couldn't quickly verify it. I did see
> > an example where the Python version has been shown to be encoded
> > inside a *.pyc file. I'm not sure how Python handles version
> > mismatches. I'm expecting that it reads the file, deserializes it,
> > check the version and if a mismatch is detected, it discards the *.pyc
> > file and compiles the *.py file. This would mean that a *.pyc file is
> > always useless for one of the two installed Python versions.
> 
> Thank you for the research and abstract. All this is not very convincing
> (beware, I'm not talking about you). I remember a discussion that I had
> in a distant past with Philip Brown about Emacs Lisp compiled files (.el is the source,
> .elc is the compiled version); after a lot of wrangling I won by the
> missing argument for architectural / version specific compiled
> components and eventually delivered them in the package as it seems
> suitable from my stand point for Python.

I found a quote from Python docs: 
http://docs.python.org/2/glossary.html#term-bytecode

"""Do note that bytecodes are not expected to work between different Python
virtual machines, nor to be stable between Python releases."""

Just to add weight to the argument that .pyc files are not portable
across Python releases (e.g. 2.6 and 2.7).

What are you thinking of doing then, delivering bytecode for one
specific python version and rendering the *.pyc files useless for other
versions?

> > I looked at the Debian pyshared solution by poking an installed
> > system. The quick summary is:
> >
> > - /usr/share/pyshared contains the .py files and is not a member of sys.path
> > - for each Python version a /usr/lib/pythonX.Y/dist-modules directory
> > exists and is a member of sys.path
> > - when a module is installed, symlinks from
> > /usr/lib/pythonX.Y/dist-modules to respective entries in
> > /usr/share/pyshared are made. They are exclusively file symlinks, and
> > never directory symlinks.
> > - the *.py files are compiled once per Python version, and *.pyc files
> > live in the /usr/lib/pythonX.Y/dist-modules directory tree
> 
> This is a genuine example of over-engineering, especially that there is
> no real argument for doing that (see above).

If you want the modules to be shared across 2.6 and 2.7, and you already
know that you cannot share the *.pyc files across 2.6 and 2.7, why do
you see no real argument?

If we ship useless *.pyc files, we can just skip shipping them.

> Moreover, if we have separate versioned trees what's the need for
> that?

You want to share modules between 2.6 and 2.7, right? Then you need to
keep the *.py files in a shared place, and keep *.pyc files in
a non-shared place.

> > Sounds very reasonable to me. Somebody has to implement it for us though.
> 
> IMHO we don't have the resources for that. However, someone willing to
> spend his time on it is free to do it.
> 
> In the mean time, what we must do is to find a solution for the
> compilation errors during the installation of some Python modules, as
> reported in the original message.

The pycompile classaction must be aware of the Python version for which
the module is built and invoke the right interpreter.

I have to say these errors seem to disprove the "2.6 and 2.7 are totally
compatible" claim. Or is there a different (than the file contents)
reason for these failures, do you know?

Maciej