From dam at opencsw.org Mon Jul 1 22:43:32 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Mon, 1 Jul 2013 22:43:32 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D04BA2.3070503@opencsw.org> References: <51D04BA2.3070503@opencsw.org> Message-ID: <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> Hi Laurent, Am 30.06.2013 um 17:15 schrieb Laurent Blume : > I['m trying to make a packages adding a few symlinks in /usr/lib so that > the NSS bits of SAmba can be used. > I had hoped I could just present a fully ready experimental package for > you people to look at, but I'm hitting a snag, and I can't seem to get > around it: > whatever I tried, the 64 bit part never gets added to the package. > > Here's what's in the recipe (I've tried variants): > > PKGFILES_CSWsamba-nss-system-links += /usr/lib/nss_winbind_csw.so.1 > PKGFILES_CSWsamba-nss-system-links += /usr/lib/nss_wins_csw.so.1 > PKGFILES_CSWsamba-nss-system-links += /usr/lib/amd64/nss_winbind_csw.so.1 > PKGFILES_CSWsamba-nss-system-links += /usr/lib/amd64/nss_wins_csw.so.1 > PKGFILES_CSWsamba-nss-system-links += /usr/lib/sparcv9/nss_winbind_csw.so.1 > PKGFILES_CSWsamba-nss-system-links += /usr/lib/sparcv9/nss_wins_csw.so.1 > > The symlinks have been recated: > Jun 30 16:58 install-isa-pentium_pro/usr/lib/nss_winbind_csw.so.1 -> > ../../opt/csw/lib/nss_winbind.so.1 > Jun 30 16:58 install-isa-pentium_pro/usr/lib/nss_wins_csw.so.1 -> > ../../opt/csw/lib/nss_wins.so.1 > > install-isa-amd64/usr/lib/amd64: > lrwxrwxrwx 1 laurent staff 40 Jun 30 17:04 > nss_winbind_csw.so.1 -> ../../../opt/csw/lib/64/nss_winbind.so.1 > lrwxrwxrwx 1 laurent staff 37 Jun 30 17:04 nss_wins_csw.so.1 > -> ../../../opt/csw/lib/64/nss_wins.so.1 > > But I only ever get the 32 bit ones in the prototype: > d none /opt/csw/share/doc/samba_nss_system_links 0755 root bin > f none /opt/csw/share/doc/samba_nss_system_links/license 0644 root bin > d none /usr/lib 0755 root bin > s none /usr/lib/nss_winbind_csw.so.1=../../opt/csw/lib/nss_winbind.so.1 > s none /usr/lib/nss_wins_csw.so.1=../../opt/csw/lib/nss_wins.so.1 > i checkpkg_override=checkpkg_override.CSWsamba-nss-system-links > > Any idea what I'm missing there? It's quite annoying. Sure :-) It goes like this: - there are versions built for 32 and 64 bit built in different directories like work//build-isa- - these are then merged together to work//pkgroot - from there the files for the packages are picked Please see also my "Advanced mGAR" talk at http://sourceforge.net/apps/trac/gar/wiki/Learning%20the%20details Inspection shows that the files are already missing in pkgroot/, so the problem is in the merge phase. The relevant definitions are beginning at http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.mk#L772 Per default only stuff from bin/, sbin/, lib/ and libexec/ (all in /opt/csw) is propagated for a 64 bit build, but not /usr/lib. If you also want /usr/lib you need something like EXTRA_MERGE_DIRS_isa-extra += /usr/lib I have not tested it, please give it a try, if it doesn't work I'll have a deeper look, then please mail your directory on the buildfarm in your home directory. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From laurent at opencsw.org Mon Jul 1 23:45:14 2013 From: laurent at opencsw.org (Laurent Blume) Date: Mon, 01 Jul 2013 23:45:14 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> References: <51D04BA2.3070503@opencsw.org> <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> Message-ID: <51D1F86A.8050506@opencsw.org> On 01/07/2013 22:43, Dagobert Michelsen wrote: > Sure :-) It goes like this: > > - there are versions built for 32 and 64 bit built in different directories like > work//build-isa- > - these are then merged together to work//pkgroot > - from there the files for the packages are picked > > Please see also my "Advanced mGAR" talk at > http://sourceforge.net/apps/trac/gar/wiki/Learning%20the%20details > > Inspection shows that the files are already missing in pkgroot/, so the problem is > in the merge phase. The relevant definitions are beginning at > http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.mk#L772 > Per default only stuff from bin/, sbin/, lib/ and libexec/ (all in /opt/csw) > is propagated for a 64 bit build, but not /usr/lib. If you also want /usr/lib > you need something like > EXTRA_MERGE_DIRS_isa-extra += /usr/lib > > I have not tested it, please give it a try, if it doesn't work I'll have a deeper > look, then please mail your directory on the buildfarm in your home directory. It's going somewhere :-) I tried your line, then this: EXTRA_MERGE_DIRS_isa-extra += /usr/lib/amd64 EXTRA_MERGE_DIRS_isa-extra += /usr/lib/sparcv9 The result is just a iittle off: work/install-isa-amd64/usr work/install-isa-amd64/usr/lib work/install-isa-amd64/usr/lib/amd64 work/install-isa-amd64/usr/lib/amd64/nss_wins_csw.so.1 work/install-isa-amd64/usr/lib/amd64/nss_winbind_csw.so.1 work/pkgroot/usr work/pkgroot/usr/lib work/pkgroot/usr/lib/amd64 work/pkgroot/usr/lib/amd64/amd64 work/pkgroot/usr/lib/amd64/amd64/nss_wins_csw.so.1 work/pkgroot/usr/lib/amd64/amd64/nss_winbind_csw.so.1 Should I just install in /usr/lib even in 64 bit? It feels a little wrong... Laurent From dam at opencsw.org Tue Jul 2 10:54:49 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 2 Jul 2013 10:54:49 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D1F86A.8050506@opencsw.org> References: <51D04BA2.3070503@opencsw.org> <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> <51D1F86A.8050506@opencsw.org> Message-ID: Hi Laurent, Am 01.07.2013 um 23:45 schrieb Laurent Blume : > On 01/07/2013 22:43, Dagobert Michelsen wrote: >> Sure :-) It goes like this: >> >> - there are versions built for 32 and 64 bit built in different directories like >> work//build-isa- >> - these are then merged together to work//pkgroot >> - from there the files for the packages are picked >> >> Please see also my "Advanced mGAR" talk at >> http://sourceforge.net/apps/trac/gar/wiki/Learning%20the%20details >> >> Inspection shows that the files are already missing in pkgroot/, so the problem is >> in the merge phase. The relevant definitions are beginning at >> http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.mk#L772 >> Per default only stuff from bin/, sbin/, lib/ and libexec/ (all in /opt/csw) >> is propagated for a 64 bit build, but not /usr/lib. If you also want /usr/lib >> you need something like >> EXTRA_MERGE_DIRS_isa-extra += /usr/lib >> >> I have not tested it, please give it a try, if it doesn't work I'll have a deeper >> look, then please mail your directory on the buildfarm in your home directory. > > It's going somewhere :-) > > I tried your line, then this: > EXTRA_MERGE_DIRS_isa-extra += /usr/lib/amd64 > EXTRA_MERGE_DIRS_isa-extra += /usr/lib/sparcv9 Ah, I see. The normal case is stuff is installed to /usr/lib/64 which is then relocated during merge. This is what merge-copy-relocated-only does: http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.mk#L947 > The result is just a iittle off: > > work/install-isa-amd64/usr > work/install-isa-amd64/usr/lib > work/install-isa-amd64/usr/lib/amd64 > work/install-isa-amd64/usr/lib/amd64/nss_wins_csw.so.1 > work/install-isa-amd64/usr/lib/amd64/nss_winbind_csw.so.1 > work/pkgroot/usr > work/pkgroot/usr/lib > work/pkgroot/usr/lib/amd64 > work/pkgroot/usr/lib/amd64/amd64 > work/pkgroot/usr/lib/amd64/amd64/nss_wins_csw.so.1 > work/pkgroot/usr/lib/amd64/amd64/nss_winbind_csw.so.1 > > > Should I just install in /usr/lib even in 64 bit? It feels a little wrong? You should install in /usr/lib/64 and set EXTRA_MERGE_DIRS_isa-extra += /usr/lib This will relocate /usr/lib/64 to /usr/lib/amd64 and /usr/lib/sparcv9. I know this is complex, but I am only responsible for the implementation, not the concept ;-) Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From dam at opencsw.org Tue Jul 2 12:00:03 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 2 Jul 2013 12:00:03 +0200 Subject: [csw-maintainers] Update of flex Message-ID: Fellow maintainers, I took the opportunity to obsolete flex 2.5.4a, drop flex_new and update flex to 2.5.37. The new packages will appear on the mirrors soon and I will then update the buildfarm. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From laurent at opencsw.org Tue Jul 2 12:07:39 2013 From: laurent at opencsw.org (Laurent Blume) Date: Tue, 02 Jul 2013 12:07:39 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: References: <51D04BA2.3070503@opencsw.org> <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> <51D1F86A.8050506@opencsw.org> Message-ID: <51D2A66B.8040101@opencsw.org> On 02/07/13 10:54, Dagobert Michelsen wrote: > You should install in /usr/lib/64 and set > EXTRA_MERGE_DIRS_isa-extra += /usr/lib > > This will relocate /usr/lib/64 to /usr/lib/amd64 and /usr/lib/sparcv9. > > I know this is complex, but I am only responsible for the implementation, > not the concept ;-) Haha, no, it's quite logical, actually I had tried first with lib/64, then when it was not working, used the arch names. Okay, we've almost got a winner there: $ more CSWsamba-nss-system-links.prototype d none /opt/csw/share/doc/samba_nss_system_links 0755 root bin f none /opt/csw/share/doc/samba_nss_system_links/license 0644 root bin d none /usr/lib 0755 root bin d none /usr/lib/amd64 0755 root bin s none /usr/lib/amd64/nss_winbind_csw.so.1=../../../opt/csw/lib/64/nss_winbind.so.1 s none /usr/lib/amd64/nss_wins_csw.so.1=../../../opt/csw/lib/64/nss_wins.so.1 s none /usr/lib/nss_winbind_csw.so.1=../../opt/csw/lib/nss_winbind.so.1 s none /usr/lib/nss_wins_csw.so.1=../../opt/csw/lib/nss_wins.so.1 i checkpkg_override=checkpkg_override.CSWsamba-nss-system-links One little thing: is it possible to avoid including the directories? /usr/lib, /usr/lib/amd64? To avoid any potential conflict with the system's idea of what it should be. Laurent From dam at opencsw.org Tue Jul 2 12:17:15 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 2 Jul 2013 12:17:15 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D2A66B.8040101@opencsw.org> References: <51D04BA2.3070503@opencsw.org> <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> <51D1F86A.8050506@opencsw.org> <51D2A66B.8040101@opencsw.org> Message-ID: <90E350B2-86D5-4999-9346-D8D79B4AB78F@opencsw.org> Hi Laurent, Am 02.07.2013 um 12:07 schrieb Laurent Blume : > On 02/07/13 10:54, Dagobert Michelsen wrote: >> You should install in /usr/lib/64 and set >> EXTRA_MERGE_DIRS_isa-extra += /usr/lib >> >> This will relocate /usr/lib/64 to /usr/lib/amd64 and /usr/lib/sparcv9. >> >> I know this is complex, but I am only responsible for the implementation, >> not the concept ;-) > > Haha, no, it's quite logical, actually I had tried first with lib/64, then when it was not working, used the arch names. > > Okay, we've almost got a winner there: > > $ more CSWsamba-nss-system-links.prototype > d none /opt/csw/share/doc/samba_nss_system_links 0755 root bin > f none /opt/csw/share/doc/samba_nss_system_links/license 0644 root bin > d none /usr/lib 0755 root bin > d none /usr/lib/amd64 0755 root bin > s none /usr/lib/amd64/nss_winbind_csw.so.1=../../../opt/csw/lib/64/nss_winbind.so.1 > s none /usr/lib/amd64/nss_wins_csw.so.1=../../../opt/csw/lib/64/nss_wins.so.1 > s none /usr/lib/nss_winbind_csw.so.1=../../opt/csw/lib/nss_winbind.so.1 > s none /usr/lib/nss_wins_csw.so.1=../../opt/csw/lib/nss_wins.so.1 > i checkpkg_override=checkpkg_override.CSWsamba-nss-system-links > > One little thing: is it possible to avoid including the directories? /usr/lib, /usr/lib/amd64? To avoid any potential conflict with the system's idea of what it should be. Hum, IIRC there is no hook for that yet. The usual place for exclusion is http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.pkg.mk#L509 but commondirs is something I am not really with anyway and as commondirs will go away in "bratislava" anyway there is change ahead. You may want to try EXTRA_PKGFILES_EXCLUDED in the pathfilter section http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.pkg.mk#L513 with something like EXTRA_PKGFILES_EXCLUDED += /usr/lib but I am not sure if this really works as the filtering is quite complex. Just drop me a note if it doesn't and I do some further analysis. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From laurent at opencsw.org Tue Jul 2 15:05:43 2013 From: laurent at opencsw.org (Laurent Blume) Date: Tue, 02 Jul 2013 15:05:43 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: References: <51D04BA2.3070503@opencsw.org> <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> <51D1F86A.8050506@opencsw.org> Message-ID: <51D2D027.5040102@opencsw.org> On 02/07/13 10:54, Dagobert Michelsen wrote: > You should install in /usr/lib/64 and set > EXTRA_MERGE_DIRS_isa-extra += /usr/lib > > This will relocate /usr/lib/64 to /usr/lib/amd64 and /usr/lib/sparcv9. > > I know this is complex, but I am only responsible for the implementation, > not the concept ;-) I take it back: it worked -once- only. Can't reproduce it, after remerge reinstall packahe, clean package, I get that: work/install-isa-amd64/usr work/install-isa-amd64/usr/lib work/install-isa-amd64/usr/lib/64 work/install-isa-amd64/usr/lib/64/nss_wins_csw.so.1 work/install-isa-amd64/usr/lib/64/nss_winbind_csw.so.1 work/install-isa-pentium_pro/usr work/install-isa-pentium_pro/usr/lib work/install-isa-pentium_pro/usr/lib/nss_wins_csw.so.1 work/install-isa-pentium_pro/usr/lib/nss_winbind_csw.so.1 work/pkgroot/usr work/pkgroot/usr/lib work/pkgroot/usr/lib/nss_wins_csw.so.1 work/pkgroot/usr/lib/nss_winbind_csw.so.1 work/pkgroot/usr/lib/amd64 work/pkgroot/usr/lib/amd64/64 work/pkgroot/usr/lib/amd64/64/nss_wins_csw.so.1 work/pkgroot/usr/lib/amd64/64/nss_winbind_csw.so.1 There's something fishy there. Laurent From pfelecan at opencsw.org Tue Jul 2 16:18:19 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 02 Jul 2013 16:18:19 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D04BA2.3070503@opencsw.org> (Laurent Blume's message of "Sun, 30 Jun 2013 17:15:46 +0200") References: <51D04BA2.3070503@opencsw.org> Message-ID: Out of curiosity: why do you install stuff in /usr/lib? I thought that our standards preclude this... -- Peter From laurent at opencsw.org Tue Jul 2 18:26:18 2013 From: laurent at opencsw.org (Laurent Blume) Date: Tue, 02 Jul 2013 18:26:18 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: References: <51D04BA2.3070503@opencsw.org> Message-ID: <51D2FF2A.3010501@opencsw.org> On 2013-07-02 4:18 PM, Peter FELECAN wrote: > Out of curiosity: why do you install stuff in /usr/lib? I thought that > our standards preclude this... At this point, it's still experimental: I'm not sure yet I'll push it (if I get it to work). Those files are used in nsswitch.conf. There's not much choice there: if you need to use them, they must be in /usr/lib. The idea is to put files in /opt/csw only, with a separate package with only symlinks to make it an easy, optional installation for those who want it. Better than let them mess with symlinks. But obviously, the standard is too well respected, and there's not enough practice of doing that, there are snags ;-) Laurent From pfelecan at opencsw.org Tue Jul 2 20:32:46 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 02 Jul 2013 20:32:46 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D2FF2A.3010501@opencsw.org> (Laurent Blume's message of "Tue, 02 Jul 2013 18:26:18 +0200") References: <51D04BA2.3070503@opencsw.org> <51D2FF2A.3010501@opencsw.org> Message-ID: Laurent Blume writes: > On 2013-07-02 4:18 PM, Peter FELECAN wrote: >> Out of curiosity: why do you install stuff in /usr/lib? I thought that >> our standards preclude this... > > At this point, it's still experimental: I'm not sure yet I'll push it > (if I get it to work). > > Those files are used in nsswitch.conf. There's not much choice there: if > you need to use them, they must be in /usr/lib. > The idea is to put files in /opt/csw only, with a separate package with > only symlinks to make it an easy, optional installation for those who > want it. Better than let them mess with symlinks. I understand. Thank you for the explanation. However, it's not possible to put this files in the required place by extra-mgar, e.g., postinstall? > But obviously, the standard is too well respected, and there's not > enough practice of doing that, there are snags ;-) I'm very comfortable with this robustness... -- Peter From laurent at opencsw.org Tue Jul 2 20:37:51 2013 From: laurent at opencsw.org (Laurent Blume) Date: Tue, 02 Jul 2013 20:37:51 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> References: <51D04BA2.3070503@opencsw.org> <248270B8-CEDD-4DB6-ABED-8C234706A2C3@opencsw.org> Message-ID: <51D31DFF.1020703@opencsw.org> On 2013-07-01 10:43 PM, Dagobert Michelsen wrote: > EXTRA_MERGE_DIRS_isa-extra += /usr/lib Okay, this works only if the symlinks are installed in /usr/lib for 64 bit too. AFAICT, the root cause is this line: http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.mk#L942 The /usr/lib bit is replaced by /usr/lib/$ISA, which can't work if there's a /64 after that. The /opt/csw dirs at this point do include the $ISA bit already. I'm honestly not sure what's the best way to do it, but installing to /usr/lib just seems easier for now. I was confused the first time because I had used the wrong remerge/reinstall combination. This one works after a spotless. Laurent From laurent at opencsw.org Tue Jul 2 20:58:53 2013 From: laurent at opencsw.org (Laurent Blume) Date: Tue, 02 Jul 2013 20:58:53 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: References: <51D04BA2.3070503@opencsw.org> <51D2FF2A.3010501@opencsw.org> Message-ID: <51D322ED.1010504@opencsw.org> On 2013-07-02 8:32 PM, Peter FELECAN wrote: > I understand. Thank you for the explanation. However, it's not possible > to put this files in the required place by extra-mgar, e.g., > postinstall? That's the thing, I want them to be optional. IF putting them in a postinstall, it means they'll be installed as part of something else. I'd see that as a workaround, if messing with /usr, it might as well be with a clean package. > I'm very comfortable with this robustness... Haha, well, there are a couple other packages that try to do things in /usr already, and I'm not sure they still work. /usr should be avoided at all costs, agreed. But it can't be completely ignored :-) Laurent From bwalton at opencsw.org Tue Jul 2 22:33:06 2013 From: bwalton at opencsw.org (Ben Walton) Date: Tue, 2 Jul 2013 21:33:06 +0100 Subject: [csw-maintainers] ruby packaging Message-ID: Hi All, Does anyone have interest in pickup up the ruby packages? I don't have the time to give them the attention they need. If anyone else (maybe some of the puppet crowd) had the time to take this, I'd appreciate it. This is the last set of packages holding libssl 0.9.8 in the catalog, btw. Thanks -Ben From pfelecan at opencsw.org Wed Jul 3 08:51:56 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Wed, 03 Jul 2013 08:51:56 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D322ED.1010504@opencsw.org> (Laurent Blume's message of "Tue, 02 Jul 2013 20:58:53 +0200") References: <51D04BA2.3070503@opencsw.org> <51D2FF2A.3010501@opencsw.org> <51D322ED.1010504@opencsw.org> Message-ID: Laurent Blume writes: > On 2013-07-02 8:32 PM, Peter FELECAN wrote: >> I'm very comfortable with this robustness... > > Haha, well, there are a couple other packages that try to do things in > /usr already, and I'm not sure they still work. They probably predate the current state of the mentioned robustness implementation. > /usr should be avoided at all costs, agreed. But it can't be completely > ignored :-) Avoidance is not ignorance. But, sure, if nss requires its elements in /usr/lib then go for it. Sincerely, I don't understand your reluctance to solve this through a post-install kludge. If it's thus solvable, modifying mgar to support this is IMHO over-engineering. -- Peter From grzemba at contac-dt.de Wed Jul 3 09:04:37 2013 From: grzemba at contac-dt.de (Carsten Grzemba) Date: Wed, 03 Jul 2013 09:04:37 +0200 Subject: [csw-maintainers] Build a local cache datebase on private buildfarm In-Reply-To: References: Message-ID: Hi Maciej, is http://wiki.opencsw.org/checkpkg#toc5 still the right reference for create a local cache database?: ... mkp: exec( rm -rf /home/cgrzemba/spool.5.10-sparc/CSWlibgstvideo0-10-0 ) WARNING:root:Cache database is not up to date. Refreshing it. WARNING:root:Refreshing the database. It may take a long time, please be patient. WARNING:root:If you need a way to make it faster, please see: WARNING:root:http://wiki.opencsw.org/checkpkg#toc5 Carsten -------------- next part -------------- An HTML attachment was scrubbed... URL: From laurent at opencsw.org Wed Jul 3 11:13:21 2013 From: laurent at opencsw.org (Laurent Blume) Date: Wed, 03 Jul 2013 11:13:21 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: References: <51D04BA2.3070503@opencsw.org> <51D2FF2A.3010501@opencsw.org> <51D322ED.1010504@opencsw.org> Message-ID: <51D3EB31.9070407@opencsw.org> On 03/07/13 08:51, Peter FELECAN wrote: > Avoidance is not ignorance. But, sure, if nss requires its elements in > /usr/lib then go for it. Sincerely, I don't understand your reluctance > to solve this through a post-install kludge. If it's thus solvable, > modifying mgar to support this is IMHO over-engineering. I just don't see the benefit of a script, and I do see an issue (being non optional as part of another package). I like stuff to be properly referenced by pkginfo, unless it's an editable configuration file. Installing a new package, I would certainly prefer it using proper system tools to modify /usr than having a script go tinker with it (and have to deal with -R, zones, whatever). Why bother? There's packaging already. Why does it shock you to use a package to install things in /usr? AFAICT, mgar *should* support it already, since the mechanism is there. It's just either a little underdocumented or not quite polished. So that should be clarified. Laurent From pfelecan at opencsw.org Wed Jul 3 11:41:06 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Wed, 03 Jul 2013 11:41:06 +0200 Subject: [csw-maintainers] Adding files from /usr/lib/64/ won't work In-Reply-To: <51D3EB31.9070407@opencsw.org> (Laurent Blume's message of "Wed, 03 Jul 2013 11:13:21 +0200") References: <51D04BA2.3070503@opencsw.org> <51D2FF2A.3010501@opencsw.org> <51D322ED.1010504@opencsw.org> <51D3EB31.9070407@opencsw.org> Message-ID: Laurent Blume writes: > On 03/07/13 08:51, Peter FELECAN wrote: >> Avoidance is not ignorance. But, sure, if nss requires its elements in >> /usr/lib then go for it. Sincerely, I don't understand your reluctance >> to solve this through a post-install kludge. If it's thus solvable, >> modifying mgar to support this is IMHO over-engineering. > > I just don't see the benefit of a script, and I do see an issue (being > non optional as part of another package). I like stuff to be properly > referenced by pkginfo, unless it's an editable configuration > file. Installing a new package, I would certainly prefer it using > proper system tools to modify /usr than having a script go tinker with > it (and have to deal with -R, zones, whatever). Why bother? There's > packaging already. Why does it shock you to use a package to install > things in /usr? You can use installf(1M) to take into account the components thus installed. It doesn't shock me per se. If it's exceptional it should be treated exceptionally. What I propose is just that kind of treatment. > AFAICT, mgar *should* support it already, since the mechanism is > there. It's just either a little underdocumented or not quite > polished. So that should be clarified. It's up to whoever maintains that to decide if the effort required to enhance that is worth the bother. -- Peter From maciej at opencsw.org Wed Jul 3 17:36:57 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Wed, 3 Jul 2013 16:36:57 +0100 Subject: [csw-maintainers] Build a local cache datebase on private buildfarm In-Reply-To: References: Message-ID: 2013/7/3 Carsten Grzemba : > is http://wiki.opencsw.org/checkpkg#toc5 still the right reference for > create a local cache database?: If you're using the current code in Subversion, it is, but I've been working on a rewrite, and I'd be obliged if you tried the new code! The instructions are here: http://wiki.opencsw.org/checkpkg#toc25 The new code is more stable than the old code in terms of being able to index the whole catalog in a single run without running out of memory. I've mostly finished the development cycle (read: introducing bugs), and I'm currently looking for more testing and correcting any issues. It runs fine on my VM, but the code really needs to be tested by people who are not me. :-) Maciej From dam at opencsw.org Thu Jul 4 14:15:30 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Thu, 4 Jul 2013 14:15:30 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ Message-ID: Hi folks, I was just trying to compile filezilla again now that have a recent wxwidgets and noticed that it requires g++ as Sun Studio is lacking some features. But wxwidgets was compiled with Sun Studio and so the mangled C++ symbols can not be resolved during linking. I guess we should really make plans for the bratislava transition. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From laurent at opencsw.org Thu Jul 4 14:33:14 2013 From: laurent at opencsw.org (Laurent Blume) Date: Thu, 04 Jul 2013 14:33:14 +0200 Subject: [csw-maintainers] New and Improved: Samba Message-ID: <51D56B8A.7090504@opencsw.org> Hello all, I've just pushed an experimental Samba 3.6.16 there: http://buildfarm.opencsw.org/experimental.html#samba I'm hoping to make it the new default soon. Here are the changes, not huge ones, but useful: - include patch for group sorting: https://bugzilla.samba.org/show_bug.cgi?id=7588 Not a Samba bug per se, but a limitation of S10/OI that the patch works around; - Rename nss modules to the Solaris standard (nss_name.so.1), rather than the inadequate libnss_name.so.1 that was used before; - Add nss_wins.so.1 (following Solaris Samba there, it probably had been overlooked); - Since they're not libraries, and the .1 is defined by the system, both above are grouped into a more neutral CSWsamba-nss package, replacing CSWlibnss-winbind1; - Create system symlinks, to make it more usable: the NSS modules above are used in nsswitch,conf, which will look in /usr/lib and /usr/lib/64 only, also, PAM modules in pam.conf will be looked for in /usr/lib/security and /usr/lib/security/64. To avoid the users tinkering with those, I made two new special packages containing only symlinks: CSWsamba-nss-system-links & CSWsamba-pam-system-links They create /usr/lib/nss_winbind_csw.so.1, /usr/lib/security/pam_winbind_csw.so, and so on. The links are relative, so they can be used properly with different rootpaths. They add the _csw suffix to the name to avoid conflicting with the Solaris packages there. To use NSS, they'll be winbind_csw and wins_csw in nsswitch.conf, to use PAM, pam_winbind_csw.so and pam_smbpass_csw,so in pam.conf. Who is using any of those to validate all? I can't test all of it myself, I don't have the infrastructure :-) Cheers, Laurent From pfelecan at opencsw.org Thu Jul 4 15:00:42 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 04 Jul 2013 15:00:42 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: (Dagobert Michelsen's message of "Thu, 4 Jul 2013 14:15:30 +0200") References: Message-ID: Dagobert Michelsen writes: > I was just trying to compile filezilla again now that have a recent wxwidgets > and noticed that it requires g++ as Sun Studio is lacking some features. > But wxwidgets was compiled with Sun Studio and so the mangled C++ symbols > can not be resolved during linking. I guess we should really make plans > for the bratislava transition. I think that the kind of packages as wxwidgets, which are used by many other project should be built with g++. If there is no relevant performance gain building with SS please use g++. We discussed this already, even during the new wxwidgets packaging but there is still reluctance using g++. Can we make a statement about this please? -- Peter From laurent at opencsw.org Thu Jul 4 15:10:33 2013 From: laurent at opencsw.org (Laurent Blume) Date: Thu, 04 Jul 2013 15:10:33 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: References: Message-ID: <51D57449.3020302@opencsw.org> On 04/07/13 15:00, Peter FELECAN wrote: > I think that the kind of packages as wxwidgets, which are used by many > other project should be built with g++. If there is no relevant > performance gain building with SS please use g++. We discussed this > already, even during the new wxwidgets packaging but there is still > reluctance using g++. Can we make a statement about this please? I think the only question is about the performance of g++ on the sparc platform. I don't know if there's still a significant difference there. As for switching ABIs: wxwidget is a very good candidate at this point since we just rebuilt (and fixed!) all its dependencies, and there are not many. I propose to try the g++ compatible ABI for Studio on the x86 version, and use g++ on sparc (to be switched back to Studio w/ g++ compatibility when it becomes available). How does that sound? Laurent From pfelecan at opencsw.org Thu Jul 4 15:38:18 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 04 Jul 2013 15:38:18 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: <51D57449.3020302@opencsw.org> (Laurent Blume's message of "Thu, 04 Jul 2013 15:10:33 +0200") References: <51D57449.3020302@opencsw.org> Message-ID: Laurent Blume writes: > On 04/07/13 15:00, Peter FELECAN wrote: >> I think that the kind of packages as wxwidgets, which are used by many >> other project should be built with g++. If there is no relevant >> performance gain building with SS please use g++. We discussed this >> already, even during the new wxwidgets packaging but there is still >> reluctance using g++. Can we make a statement about this please? > > I think the only question is about the performance of g++ on the sparc > platform. I don't know if there's still a significant difference > there. How can we test the performance on SPARC between programs built with these compilers? Have you some references about this issue? Is there a benchmark used to substantiate it? > As for switching ABIs: wxwidget is a very good candidate at this point > since we just rebuilt (and fixed!) all its dependencies, and there are > not many. > I propose to try the g++ compatible ABI for Studio on the x86 version, > and use g++ on sparc (to be switched back to Studio w/ g++ > compatibility when it becomes available). How does that sound? In what you propose, we still use g++ on the very platform which has a performance issue and with SS on the platform where g++ doesn't. Why not go with g++ on all platforms in this case? Maintenance wise is less complex, isn't it? -- Peter From laurent at opencsw.org Thu Jul 4 15:49:27 2013 From: laurent at opencsw.org (Laurent Blume) Date: Thu, 04 Jul 2013 15:49:27 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: References: <51D57449.3020302@opencsw.org> Message-ID: <51D57D67.7000807@opencsw.org> On 04/07/13 15:38, Peter FELECAN wrote: > How can we test the performance on SPARC between programs built with > these compilers? > > Have you some references about this issue? Is there a benchmark used to > substantiate it? I used to bench it using "openssl speed". That was a long time ago, and Studio was definitely much faster then. I've not done it recently. > In what you propose, we still use g++ on the very platform which has a > performance issue and with SS on the platform where g++ doesn't. Why not > go with g++ on all platforms in this case? Maintenance wise is less > complex, isn't it? Yes, I agree. I see it mostly as a way to try it, even if sadly, it's not possible to try it where it'd be most needed. And as I'm maintaining that particular bit, it's not just something I'm pushing on somebody else. The goal is to provide wxwidget with g++ ABI to others. As to how I do it, they should not worry about that ;-) Laurent From pfelecan at opencsw.org Thu Jul 4 15:58:43 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 04 Jul 2013 15:58:43 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: <51D57D67.7000807@opencsw.org> (Laurent Blume's message of "Thu, 04 Jul 2013 15:49:27 +0200") References: <51D57449.3020302@opencsw.org> <51D57D67.7000807@opencsw.org> Message-ID: Laurent Blume writes: > On 04/07/13 15:38, Peter FELECAN wrote: >> How can we test the performance on SPARC between programs built with >> these compilers? >> >> Have you some references about this issue? Is there a benchmark used to >> substantiate it? > > I used to bench it using "openssl speed". That was a long time ago, > and Studio was definitely much faster then. I've not done it recently. Can you elaborate a bit on "openssl speed"? You agree that we are not talking about SS speed but the speed of binaries built with SS, isn't it? >> In what you propose, we still use g++ on the very platform which has a >> performance issue and with SS on the platform where g++ doesn't. Why not >> go with g++ on all platforms in this case? Maintenance wise is less >> complex, isn't it? > > Yes, I agree. I see it mostly as a way to try it, even if sadly, it's > not possible to try it where it'd be most needed. > And as I'm maintaining that particular bit, it's not just something > I'm pushing on somebody else. The goal is to provide wxwidget with g++ > ABI to others. As to how I do it, they should not worry about that ;-) You're right, consequently I agree... -- Peter From laurent at opencsw.org Thu Jul 4 16:46:51 2013 From: laurent at opencsw.org (Laurent Blume) Date: Thu, 04 Jul 2013 16:46:51 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: References: <51D57449.3020302@opencsw.org> <51D57D67.7000807@opencsw.org> Message-ID: <51D58ADB.50602@opencsw.org> On 04/07/13 15:58, Peter FELECAN wrote: > Can you elaborate a bit on "openssl speed"? You agree that we are not > talking about SS speed but the speed of binaries built with SS, isn't > it? Yup, it's a standard openssl command. Just type "openssl speed", it gives you a nice bench. I've found it useful to compare raw binary efficiency and also the effect of different compiler optimization settings. One interesting thing is that not all algorithms tested get the same results, so it's a rather comprehensive test. Ie, when I tested on x86, some algorithms performed better with GCC, others with Studio, but overall, Studio was superior. Laurent From jh at opencsw.org Thu Jul 4 17:28:07 2013 From: jh at opencsw.org (Jan Holzhueter) Date: Thu, 04 Jul 2013 17:28:07 +0200 Subject: [csw-maintainers] pkgutil option to install lower OS Version package Message-ID: <51D59487.7090208@opencsw.org> Hi, just a thought I just had. I know we should smack people with Solaris 10 Version beeing older then 5 years, but since it happens 1-2 times a month how about adding an option to force to install the Solaris 9 cataloge instead of the Solaris 10? Greetings Jan From maciej at opencsw.org Thu Jul 4 18:02:48 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Thu, 4 Jul 2013 17:02:48 +0100 Subject: [csw-maintainers] pkgutil option to install lower OS Version package In-Reply-To: <51D59487.7090208@opencsw.org> References: <51D59487.7090208@opencsw.org> Message-ID: 2013/7/4 Jan Holzhueter > > I know we should smack people with Solaris 10 Version beeing older then > 5 years, but since it happens 1-2 times a month how about adding an > option to force to install the Solaris 9 cataloge instead of the Solaris > 10? You could achieve the same effect by simply symlinking the e.g. sparc/5.10 to sparc/5.9. The question is where do you make that symlink. I'm thinking that you could make a local mirror and then selectively symlink directories you want to where you want them. Maciej From pfelecan at opencsw.org Thu Jul 4 19:20:07 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 04 Jul 2013 19:20:07 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: <51D58ADB.50602@opencsw.org> (Laurent Blume's message of "Thu, 04 Jul 2013 16:46:51 +0200") References: <51D57449.3020302@opencsw.org> <51D57D67.7000807@opencsw.org> <51D58ADB.50602@opencsw.org> Message-ID: Laurent Blume writes: > On 04/07/13 15:58, Peter FELECAN wrote: >> Can you elaborate a bit on "openssl speed"? You agree that we are not >> talking about SS speed but the speed of binaries built with SS, isn't >> it? > > Yup, it's a standard openssl command. Just type "openssl speed", it > gives you a nice bench. I've found it useful to compare raw binary > efficiency and also the effect of different compiler optimization > settings. One interesting thing is that not all algorithms tested get > the same results, so it's a rather comprehensive test. Ie, when I > tested on x86, some algorithms performed better with GCC, others with > Studio, but overall, Studio was superior. Maybe I misunderstood the issue but openssl is implemented mostly in C and there is no C++. Consequently, the above described benchmark is about code generation when the source is C but we are discussing g++ against SSCC as C++ compilers, wxwidgets being mainly C++. BTW, is wxwidgets in the class of libraries requiring such a great optimization, such as openssl, that it doesn't suffer a little bit of lag on SPARC? -- Peter From laurent at opencsw.org Thu Jul 4 20:55:51 2013 From: laurent at opencsw.org (Laurent Blume) Date: Thu, 04 Jul 2013 20:55:51 +0200 Subject: [csw-maintainers] Filezille, wxwidgets and C++ In-Reply-To: References: <51D57449.3020302@opencsw.org> <51D57D67.7000807@opencsw.org> <51D58ADB.50602@opencsw.org> Message-ID: <51D5C537.30700@opencsw.org> On 2013-07-04 7:20 PM, Peter FELECAN wrote: > Maybe I misunderstood the issue but openssl is implemented mostly in C > and there is no C++. Consequently, the above described benchmark is > about code generation when the source is C but we are discussing g++ > against SSCC as C++ compilers, wxwidgets being mainly C++. We are, but the C/C++ bit is really only relevant for the ABI. For binary code performance, both gcc and g++ use the same backend, as explained eg here: http://www.redhat.com/magazine/002dec04/features/gcc/#internal-org-gcc If I remember correctly, some years after that, Fortran was included too. As far as I know, Solaris Studio (and all compilers) work in the same ways, because it's simply easier to maintain a single optimization engine (or rather, to integrate several of them a single time from processor teams for Intel/AMD/SPARC64/UltraSPARC-T design). You can see it for Studio in that the patch that contains binary code fixes is "Compiler Common components", 148906-06. Those specific to C/C++ don't relate to binary optimization. So openssl is indeed useful to compare binary performance. C or C++ does not matter for that particular benchmark. > BTW, is wxwidgets in the class of libraries requiring such a great optimization, > such as openssl, that it doesn't suffer a little bit of lag on SPARC? I would say, any interactive UI lib needs as much optimization as it can :-) As for the specifics, I don't know how to bench it and if it would be perceptible or not. Any idea how to do that? Laurent From grzemba at contac-dt.de Tue Jul 9 10:05:21 2013 From: grzemba at contac-dt.de (Carsten Grzemba) Date: Tue, 09 Jul 2013 10:05:21 +0200 Subject: [csw-maintainers] checkpkg problem In-Reply-To: References: Message-ID: Hi, checkpkg miss the 64bit lib for PHP5, but I think nobody need this and I override this already. But checkpkg still suggest this to override and at last it reports an unused override???? How can I satisfy checkpkg? Carsten ?cgrzemba at login:~/pkgs$ /home/cgrzemba/opencsw/.buildsys/v2/bin/checkpkg --catalog-release unstable --os-release SunOS5.10 --catalog-architecture i386 6acb96ae98095c8fbdea2e5f3c46d37b ea8c2c1c1724c2b87996bdb201126f89 6e27859db5ae1c441f33fad1fab5594e 29114229b1e72d3be54707235245cd55 5e26565537f28f3b80c37130ebee046a 2aace9d8149bd3dd107f74c736336c3b 98ffa9d4da048e6e49056c632074cdbf f175e22c1d061f2420f2a38a9a5f7db4 5711e91841f4d216dc66a224cf215a4b 8f47334058d908167650f02744ac1bba 35fbac112b34d7e49c760b5951a22dbe INFO:root:Unwrapping candies... 100% |###############################################################################|INFO:root:Tasting candies one by one... 100% |###############################################################################|INFO:root:Tasting them all at once... INFO:root:Stuffing the candies under the pillow... 100% |###############################################################################|CSWcups: CSWphp5-cups: ?* The package contains 32-bit binaries, e.g. {'soname': 'phpcups.so', 'base_name': 'phpcups.so', 'RUNPATH RPATH the same': True, 'runpath': ['/opt/csw/lib/$ISALIST', '/opt/csw/lib', '/opt/csw/lib', '/opt/csw/lib'], 'RPATH set': True, 'needed sonames': ['libcups.so.2'], 'path': 'opt/csw/php5/lib/php/extensions/no-debug-non-zts-20090626/phpcups.so', 'RUNPATH set': True}, but it doesn't seem to contain any 64-bit binaries in the usual locations. Locations checked for 64-bit binaries: opt/csw/(lib|libexec)/(amd64). ?* Dependency issues of CSWcups: ?* If you don't know of any reasons to include these dependencies, you might remove them: ?* ? CSWcupsclient If any of the reported errors were false positives, you can override them pasting the lines below to the GAR recipe. CHECKPKG_OVERRIDES_CSWphp5-cups += 64-bit-binaries-missing Do not copy/paste these overrides without thinking! If you're not sure, scroll up and read the details. If you're still not sure, go to the wiki and read about the error tags you see, or ask on the maintainers@ mailing list. http://wiki.opencsw.org/checkpkg-error-tags WARNING: Some overrides did not match any errors. They can probably be removed, as they don't take any effect anyway. If you're getting errors at the same time, maybe you didn't specify your overrides correctly. * Unused Override: CSWphp5-cups: 64bits-binaries-missing -------------- next part -------------- An HTML attachment was scrubbed... URL: From maciej at opencsw.org Tue Jul 9 10:11:35 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 9 Jul 2013 09:11:35 +0100 Subject: [csw-maintainers] checkpkg problem In-Reply-To: References: Message-ID: 2013/7/9 Carsten Grzemba > > checkpkg miss the 64bit lib for PHP5, but I think nobody need this and I override this already. > But checkpkg still suggest this to override and at last it reports an unused override???? > > How can I satisfy checkpkg? This is a very unsophisticated check, see the message: * The package contains 32-bit binaries, e.g. {'soname': 'phpcups.so', 'base_name': 'phpcups.so', 'RUNPATH RPATH the same': True, 'runpath': ['/opt/csw/lib/$ISALIST', '/opt/csw/lib', '/opt/csw/lib', '/opt/csw/lib'], 'RPATH set': True, 'needed sonames': ['libcups.so.2'], 'path': 'opt/csw/php5/lib/php/extensions/no-debug-non-zts-20090626/phpcups.so', 'RUNPATH set': True}, but it doesn't seem to contain any 64-bit binaries in the usual locations. Locations checked for 64-bit binaries: opt/csw/(lib|libexec)/(amd64). It literally only looks into /opt/csw/(lib|libexec)/(amd64). We would need to implement a smarter version of it, to reduce the number of false positives. About the unused override, it seems to be a 2-char difference: "64bits-..." vs "64-bit-...". From maciej at opencsw.org Wed Jul 10 11:32:21 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Wed, 10 Jul 2013 10:32:21 +0100 Subject: [csw-maintainers] =?utf-8?q?Integrating_unstable=E2=86=92testing_?= =?utf-8?b?4oCUIGhhbmRvdmVy?= Message-ID: Hello maintainers, I'd like to hand over the unstable?testing integration to someone else. It's a periodic task, where you take an existing state of unstable, create a list of operations to bring testing to the same state, then you review the list and if it looks good, you execute it. If it doesn't look good (e.g. there's a broken package in unstable), you manually remove this package from the list. For example: lib/python/integrate_catalogs.py \ --from-catalog=unstable \ --to-catalog=kiel \ -o to_kiel_17.sh vim to_kiel_17.sh # Looks good? bash to_kiel_17.sh The number 17 is just there for tracking; I'm keeping the previous generated integration scripts to keep track of what I've done in the past. The task requires some tracking of the state of unstable, e.g. you need to know if unstable is in a good enough shape to be integrated/copied to testing. What's the prize? There's no prize, except for the eternal gratitude of our users. The current unstable catalog is due to integration, so if anyone wants to take it on, there's immediately something to do. Any takers? Maciej From maciej at opencsw.org Thu Jul 11 00:04:19 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Wed, 10 Jul 2013 23:04:19 +0100 Subject: [csw-maintainers] Promoting dublin to stable Message-ID: Hello maintainers, We've kept the 'stable' dead for a while now. What do you think about pointing the 'stable' symlink at dublin and the 'testing' symlink at kiel? Maciej From bonivart at opencsw.org Thu Jul 11 00:14:31 2013 From: bonivart at opencsw.org (Peter Bonivart) Date: Thu, 11 Jul 2013 00:14:31 +0200 Subject: [csw-maintainers] Promoting dublin to stable In-Reply-To: References: Message-ID: On Thu, Jul 11, 2013 at 12:04 AM, Maciej (Matchek) Blizi?ski wrote: > Hello maintainers, > > We've kept the 'stable' dead for a while now. What do you think about > pointing the 'stable' symlink at dublin and the 'testing' symlink at > kiel? +1 From pfelecan at opencsw.org Thu Jul 11 09:16:09 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 11 Jul 2013 09:16:09 +0200 Subject: [csw-maintainers] Promoting dublin to stable In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Wed, 10 Jul 2013 23:04:19 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > We've kept the 'stable' dead for a while now. What do you think about > pointing the 'stable' symlink at dublin and the 'testing' symlink at > kiel? +1 -- Peter From wilbury at opencsw.org Thu Jul 11 09:33:40 2013 From: wilbury at opencsw.org (Juraj Lutter) Date: Thu, 11 Jul 2013 09:33:40 +0200 Subject: [csw-maintainers] Promoting dublin to stable In-Reply-To: References: Message-ID: <51DE5FD4.7070201@opencsw.org> On 07/11/13 09:16, Peter FELECAN wrote: > "Maciej (Matchek) Blizi?ski" writes: > >> We've kept the 'stable' dead for a while now. What do you think about >> pointing the 'stable' symlink at dublin and the 'testing' symlink at >> kiel? > > +1 Same here. > -- Juraj Lutter From opk at opencsw.org Thu Jul 11 11:45:40 2013 From: opk at opencsw.org (Oliver Kiddle) Date: Thu, 11 Jul 2013 11:45:40 +0200 Subject: [csw-maintainers] =?utf-8?q?Integrating_unstable=E2=86=92testing_?= =?utf-8?b?4oCUIGhhbmRvdmVy?= In-Reply-To: References: Message-ID: <7058.1373535940@thecus.kiddle.eu> I'm replying to two messages here. Maciej (Matchek) Blizi?ski wrote: > We've kept the 'stable' dead for a while now. What do you think about > pointing the 'stable' symlink at dublin and the 'testing' symlink at > kiel? I'd definitely be in favour. The old stable release is way too ancient and it is a lot easier for users to understand if our naming is meaningful. To what extent have fixed packages been moved to dublin since it was frozen? Has it relied on a single person or could any maintainer run integrate_catalogs.py if they had fixed a critical issue. Maciej (Matchek) Blizi?ski wrote: > I'd like to hand over the unstable?testing integration to someone > else. It's a periodic task, where you take an existing state of > unstable, create a list of operations to bring testing to the same > state, then you review the list and if it looks good, you execute it. > If it doesn't look good (e.g. there's a broken package in unstable), > you manually remove this package from the list. I'm tempted to take on this task though in a normal situation my use of unstable can be limited. For the systems at my work, I ended up creating my own stable release by taking a snapshot from unstable as it then was and grabbing newer packages in those cases where there were problems. It was easier to use unstable as a starting point because I needed some extra things that were broken or missing at that time. This was before dublin and stable at that time was already old. We've got new hardware and I'm currently preparing a similar snapshot based on kiel. Perhaps I should use dublin but kiel seemed stable on the test machine. The result is that at the moment I'm testing and using unstable quite thoroughly. What is the intended schedule for kiel being frozen? Oliver From grzemba at contac-dt.de Thu Jul 11 13:48:52 2013 From: grzemba at contac-dt.de (Carsten Grzemba) Date: Thu, 11 Jul 2013 13:48:52 +0200 Subject: [csw-maintainers] =?windows-1252?q?ValueError=3A_seek_out_of_rang?= =?windows-1252?q?e_=97_on_import?= In-Reply-To: References: Message-ID: I haven't the problem until now. The setup of a mirror takes also some time;-) Carsten Am 11.07.13 schrieb Maciej (Matchek) Blizi?ski : > [+Carsten, +Laurent] > > I'm personally not in a hurry, I can wait while you can work on a > proper fix. I know that Carsten and Laurent were recently running the > new code, so it might be more urgent for them. > > Maciej > > 2013/7/11 Yann Rouillard : > > No I get the issue. Although I don't know yet how to properly solve it. > > You triggered this issue with another binary I suppose because I didn't have > > the problem with diskmediatest_probe.so. > > > > I will work soon on it but I am not sure I will be able to work on it before > > tonight. > > If it blocks you, you should revert to the previous elftools and make an > > exception in the code for the diskmediatest_probe.so until I properly fix > > this. > > > > Yann > > > > > > 2013/7/11 Maciej (Matchek) Blizi?ski > > > >> No, I didn't apply any patches, this is 10_U9 as released by Oracle. > >> > >> I'm getting a new error: > >> > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 124, in __init__ > >> Dynamic.__init__(self, stream, elffile, self['sh_offset']) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 66, in __init__ > >> self._find_and_set_stringtable() > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 81, in _find_and_set_stringtable > >> for section in self._elffile.iter_sections(): > >> File "/opt/csw/lib/python/site-packages/elftools/elf/elffile.py", line > >> 93, in iter_sections > >> yield self.get_section(i) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/elffile.py", line > >> 73, in get_section > >> return self._make_section(section_header) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/elffile.py", line > >> 264, in _make_section > >> return DynamicSection(section_header, name, self.stream, self) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 124, in __init__ > >> Dynamic.__init__(self, stream, elffile, self['sh_offset']) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 66, in __init__ > >> self._find_and_set_stringtable() > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 81, in _find_and_set_stringtable > >> for section in self._elffile.iter_sections(): > >> File "/opt/csw/lib/python/site-packages/elftools/elf/elffile.py", line > >> 93, in iter_sections > >> yield self.get_section(i) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/elffile.py", line > >> 73, in get_section > >> return self._make_section(section_header) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/elffile.py", line > >> 264, in _make_section > >> return DynamicSection(section_header, name, self.stream, self) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 124, in __init__ > >> Dynamic.__init__(self, stream, elffile, self['sh_offset']) > >> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", line > >> 62, in __init__ > >> self._tagsize = self._elfstructs.Elf_Dyn.sizeof() > >> File "/opt/csw/lib/python/site-packages/elftools/construct/core.py", > >> line 235, in sizeof > >> context = Container() > >> RuntimeError: maximum recursion depth exceeded > >> > >> I'm deleting the *.marshal files and running the command from scratch to > >> confirm. > >> > >> > >> 2013/7/9 Yann Rouillard > >>> > >>> Fix proposed upstream: https://github.com/eliben/pyelftools/pull/10 > >>> and applied on the opencsw package: > >>> http://sourceforge.net/apps/trac/gar/changeset/21472/csw/mgar/pkg/lang-python/pyelftools > >>> > >>> I just uploaded the package on unstable. > >>> > >>> I tested with the file and the CollectBinaryDumpInfo now works fine. > >>> > >>> Can you confirm that it is now good for you also ? > >>> > >>> BTW, did you apply a patch that updated the package containing this file > >>> ? > >>> > >>> Yann > >>> > >>> > >>> > >>> 2013/7/9 Maciej (Matchek) Blizi?ski > >>>> > >>>> Cool, thanks. > >>>> > >>>> 2013/7/9 Yann Rouillard : > >>>> > Ok I got the problem. The binary has several dynstr sections which are > >>>> > used > >>>> > to find the string name of symbols, soname... > >>>> > It seems pyelftools doesn't correctly handle this case. > >>>> > > >>>> > I am working on a fix. > >>>> > > >>>> > Yann > >>>> > > >>>> > > >>>> > 2013/7/9 Maciej (Matchek) Blizi?ski > >>>> > > >>>> >> It's interesting that the import did complete before. Here's the > >>>> >> binary. > >>>> >> > >>>> >> Maciej > >>>> >> > >>>> >> > >>>> >> 2013/7/8 Yann Rouillard > >>>> >>> > >>>> >>> Nope, I suppose there is something wrong with the binary, however I > >>>> >>> can't > >>>> >>> reproduce the problem. > >>>> >>> I tried with the /usr/sunvts/lib/probe/64/diskmediatest_probe.so > >>>> >>> file on > >>>> >>> unstable10x and the current pyelftools package. > >>>> >>> > >>>> >>> Can you send me the binary on your server and confirm that you are > >>>> >>> using > >>>> >>> the current pyelftools package ? > >>>> >>> > >>>> >>> Yann > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> 2013/7/8 Maciej (Matchek) Blizi?ski > >>>> >>> > >>>> >>>> Did you ever see this error before? It's the first time I see it. > >>>> >>>> > >>>> >>>> maciej at vsol08 ~/src/opencsw-gar $ bin/pkgdb system-metadata-to-disk > >>>> >>>> INFO:root:system-idx-SunOS5.10-i386-pkginfo.marshal already exists. > >>>> >>>> INFO:root:system-idx-SunOS5.10-i386-contents.marshal already > >>>> >>>> exists. > >>>> >>>> INFO:root:system-idx-SunOS5.10-i386-files_metadata.marshal already > >>>> >>>> exists. > >>>> >>>> INFO:root:system-idx-SunOS5.10-i386-binaries_dump_info.marshal does > >>>> >>>> not > >>>> >>>> exist. > >>>> >>>> /usr/sunvts/lib/probe/64/diskmediatest_probe.so > >>>> >>>> Traceback (most recent call last): > >>>> >>>> File "bin/pkgdb", line 718, in > >>>> >>>> main() > >>>> >>>> File "bin/pkgdb", line 547, in main > >>>> >>>> spi.IndexAndSave() > >>>> >>>> File "/home/maciej/src/opencsw-gar/lib/python/system_pkgmap.py", > >>>> >>>> line > >>>> >>>> 362, in IndexAndSave > >>>> >>>> [tuple(x) for x in self._GetBinariesDumpInfo(files_metadata)]) > >>>> >>>> File "/home/maciej/src/opencsw-gar/lib/python/system_pkgmap.py", > >>>> >>>> line > >>>> >>>> 451, in _GetBinariesDumpInfo > >>>> >>>> util.GetBinaryDumpInfo(abs_path, binary_path)) > >>>> >>>> File "/home/maciej/src/opencsw-gar/lib/python/util.py", line 107, > >>>> >>>> in > >>>> >>>> GetBinaryDumpInfo > >>>> >>>> binary_dump_info = elf_extractor.CollectBinaryDumpinfo() > >>>> >>>> File > >>>> >>>> > >>>> >>>> "/home/maciej/src/opencsw-gar/lib/python/collect_binary_elfinfo.py", line > >>>> >>>> 204, in CollectBinaryDumpinfo > >>>> >>>> for dyn_tag in sections['dynamic'].iter_tags(): > >>>> >>>> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", > >>>> >>>> line > >>>> >>>> 69, in iter_tags > >>>> >>>> tag = self.get_tag(n) > >>>> >>>> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", > >>>> >>>> line > >>>> >>>> 83, in get_tag > >>>> >>>> return DynamicTag(entry, self._elffile) > >>>> >>>> File "/opt/csw/lib/python/site-packages/elftools/elf/dynamic.py", > >>>> >>>> line > >>>> >>>> 36, in __init__ > >>>> >>>> dynstr.get_string(self.entry.d_val)) > >>>> >>>> File > >>>> >>>> "/opt/csw/lib/python/site-packages/elftools/elf/sections.py", > >>>> >>>> line 66, in get_string > >>>> >>>> s = parse_cstring_from_stream(self.stream, table_offset + > >>>> >>>> offset) > >>>> >>>> File > >>>> >>>> "/opt/csw/lib/python/site-packages/elftools/common/utils.py", > >>>> >>>> line 48, in parse_cstring_from_stream > >>>> >>>> stream.seek(stream_pos) > >>>> >>>> ValueError: seek out of range > >>>> >>>> > >>>> >>> > >>>> >> > >>>> > > >>> > >>> > >> > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maciej at opencsw.org Thu Jul 11 18:07:25 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Thu, 11 Jul 2013 17:07:25 +0100 Subject: [csw-maintainers] =?utf-8?q?Integrating_unstable=E2=86=92testing_?= =?utf-8?b?4oCUIGhhbmRvdmVy?= In-Reply-To: <7058.1373535940@thecus.kiddle.eu> References: <7058.1373535940@thecus.kiddle.eu> Message-ID: 2013/7/11 Oliver Kiddle : > I'm replying to two messages here. > > Maciej (Matchek) Blizi?ski wrote: >> We've kept the 'stable' dead for a while now. What do you think about >> pointing the 'stable' symlink at dublin and the 'testing' symlink at >> kiel? > > I'd definitely be in favour. The old stable release is way too ancient > and it is a lot easier for users to understand if our naming is > meaningful. Cool. > To what extent have fixed packages been moved to dublin since it was > frozen? In practice, close to zero updates in the last... I don't know, year? We did push PostgreSQL updates though, if I'm not mistaken. We have the capacity to submit with csw-upload-pkg to a named catalog, not only unstable. We can whitelist dublin and make submissions there a standard practice, with the regular checkpkg procedure. If anyone wants to submit packages to dublin, it's either possible already, or we can make it possible without too much effort. > Has it relied on a single person or could any maintainer run > integrate_catalogs.py if they had fixed a critical issue. Anyone can run integrate_catalogs.py, and the same goes with csw-upload-pkg. The former is far more dangerous though. > Maciej (Matchek) Blizi?ski wrote: >> I'd like to hand over the unstable?testing integration to someone >> else. It's a periodic task, where you take an existing state of >> unstable, create a list of operations to bring testing to the same >> state, then you review the list and if it looks good, you execute it. >> If it doesn't look good (e.g. there's a broken package in unstable), >> you manually remove this package from the list. > > I'm tempted to take on this task though in a normal situation my use of > unstable can be limited. > > For the systems at my work, I ended up creating my own stable release by > taking a snapshot from unstable as it then was and grabbing newer > packages in those cases where there were problems. I used to do the same. I guess it's a necessity if you run a fleet of machines. It's good that you have experience with managing catalogs. > It was easier to use > unstable as a starting point because I needed some extra things that > were broken or missing at that time. This was before dublin and stable > at that time was already old. We've got new hardware and I'm currently > preparing a similar snapshot based on kiel. Perhaps I should use dublin > but kiel seemed stable on the test machine. The result is that at the > moment I'm testing and using unstable quite thoroughly. What is the > intended schedule for kiel being frozen? There isn't a fully crystallized plan right now. There is a consensus that we will move towards timed releases, e.g. every 6 or every 12 months. With the limited workforce it has been very hard to hit our targets, but at least our efforts were coordinated. I'm thinking that what we want to finish for the kiel release is the OpenSSL migration. When we're done, we'll freeze kiel and continue with a new testing release. This topic will be definitely discussed in Berlin. We have the audacious goal of doing a full rebuild with GCC, but I still don't know if it's feasible. Maybe we should focus on IPS instead. To be discussed in Berlin. Maciej From laurent at opencsw.org Mon Jul 15 15:34:34 2013 From: laurent at opencsw.org (Laurent Blume) Date: Mon, 15 Jul 2013 15:34:34 +0200 Subject: [csw-maintainers] New ImageMagick Message-ID: <51E3FA6A.90008@opencsw.org> Hello all, I'm pushing ImageMagick 6.8.6-5. There are a few things of note: - I dropped DPS support: it's not a must-have, and it depends on a really obsolete feature of Solaris - the libs are now named like this: libMagickCore-6.Q16HDRI.so.1.0.0, that's because HDR is enabled (it already was), I don't think it should be a problem if it is linked to correctly. It also allow to have a non-HDR version if it becomes needed Cheers, Laurent From laurent at opencsw.org Wed Jul 17 10:05:49 2013 From: laurent at opencsw.org (Laurent Blume) Date: Wed, 17 Jul 2013 10:05:49 +0200 Subject: [csw-maintainers] New ImageMagick In-Reply-To: <51E3FA6A.90008@opencsw.org> References: <51E3FA6A.90008@opencsw.org> Message-ID: <51E6505D.9040400@opencsw.org> Hello all, Since we discussed that not long ago, and I've got a few thousands pics I want to integrate in a gallery, I've done a little benchmarking with ImageMagick. The conclusion is that I'm going to switch this to GCC4.8, as the performance is remarkably better, at least on AMD64. Here's the details: The system is based on an old CPU: Intel(r) Core(tm)2 Quad CPU Q9550 @ 2.83GHz I run this command on a bunch of high rez pics, a couple of times to ensure the results are consistent: export OMP_NUM_THREADS=x ptime \ bash -c "for i in _MG_10*.JPG; do /opt/csw/bin/convert "$i" -sharpen 0x1.0 -resize 80% png:/dev/null done" Here are the results, using the current build as the reference (100). Lower values are faster, and real is of course what matters most to the user: GCC4 SOS12U3 SOS12U3 -xO5 2 threads 3 threads *4 threads* *2 threads* 3 threads 4 threads 2 threads 3 threads 4 threads real 06:48,3 05:08,9 *04:30,9* *09:12,0* 06:46,8 05:48,5 07:21,3 05:37,5 04:58,2 user 11:18,7 11:34,3 *12:03,7* *15:58,2* 16:15,1 16:46,2 12:17,0 12:34,5 13:08,7 sys 00:38,2 00:44,7 *00:49,1* *00:39,3* 00:45,1 00:50,4 00:38,0 00:45,4 00:49,9 73,98 55,96 *49,08* *100,00* 73,70 63,14 79,95 61,14 54,02 70,84 72,46 *75,53* *100,00* 101,77 105,02 76,91 78,74 82,32 97,20 113,63 *124,88* *100,00* 114,58 128,22 96,70 115,30 126,89 An important thing here: the default number of threads for Studio is 2, while for GCC4, it is the number of available cores. It means that for a casual user on a 4-core system like mine, the immediate difference is really huge: the real time spent with GCC is less than half that of Studio. Even with using -xO5, Studio is still about 10% slower than GCC4.8. I tried to build using -native with SOS12U3: it crashed on an assertion failed. I also tried -O3 with GCC, but the results were about the same or even somewhat slower, so I didn't push it. I'll put packages on experimental soon, I would be /very/ interested if somebody could run a similar test on sparc hardware to compare performance (I might be able to do it on an M3000 with 16 cores). As for the ABI practicality: at this point, there's nothing depending on the C++ ImageMagick library, so it won't break anything. Future dependent packages should follow suit. Laurent Studio OpenMP variables: http://docs.oracle.com/cd/E24457_01/html/E21996/aewcb.html GCC: http://gcc.gnu.org/onlinedocs/libgomp/OMP_005fNUM_005fTHREADS.html On 15/07/13 15:34, Laurent Blume wrote: > Hello all, > > I'm pushing ImageMagick 6.8.6-5. > > There are a few things of note: > - I dropped DPS support: it's not a must-have, and it depends on a > really obsolete feature of Solaris > - the libs are now named like this: libMagickCore-6.Q16HDRI.so.1.0.0, > that's because HDR is enabled (it already was), I don't think it should > be a problem if it is linked to correctly. It also allow to have a > non-HDR version if it becomes needed > > Cheers, > > Laurent > _______________________________________________ > maintainers mailing list > maintainers at lists.opencsw.org > https://lists.opencsw.org/mailman/listinfo/maintainers > .:: This mailing list's archive is public. ::. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markp at opencsw.org Wed Jul 17 21:49:52 2013 From: markp at opencsw.org (Mark Phillips) Date: Wed, 17 Jul 2013 21:49:52 +0200 Subject: [csw-maintainers] ruby packaging In-Reply-To: References: Message-ID: On 2 Jul 2013, at 22:33, Ben Walton wrote: > Hi All, > > Does anyone have interest in pickup up the ruby packages? I don't have > the time to give them the attention they need. If anyone else (maybe > some of the puppet crowd) had the time to take this, I'd appreciate > it. The puppet 'crowd' is, umm, me. Is puppet really the only reason we maintain ruby? --Mark [freenode: phips] From bwalton at opencsw.org Thu Jul 18 00:01:57 2013 From: bwalton at opencsw.org (Ben Walton) Date: Wed, 17 Jul 2013 23:01:57 +0100 Subject: [csw-maintainers] ruby packaging In-Reply-To: References: Message-ID: I used ruby for all of my scripting previously but I'm no longer using either Solaris or ruby. Thanks -Ben On Jul 17, 2013 8:50 PM, "Mark Phillips" wrote: > On 2 Jul 2013, at 22:33, Ben Walton wrote: > > > Hi All, > > > > Does anyone have interest in pickup up the ruby packages? I don't have > > the time to give them the attention they need. If anyone else (maybe > > some of the puppet crowd) had the time to take this, I'd appreciate > > it. > > The puppet 'crowd' is, umm, me. > > Is puppet really the only reason we maintain ruby? > > --Mark [freenode: phips] > > _______________________________________________ > maintainers mailing list > maintainers at lists.opencsw.org > https://lists.opencsw.org/mailman/listinfo/maintainers > .:: This mailing list's archive is public. ::. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dam at opencsw.org Thu Jul 18 10:27:07 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Thu, 18 Jul 2013 10:27:07 +0200 Subject: [csw-maintainers] Downtime of farm on 18. and 22. July Message-ID: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> Hi folks, our power company will have a 2 hour power shutdown on the nights between 18. and 19. July and between 22. and 23. July. As our UPS doesn't stand this long we are shutting down the machines in the evening and boot everything up in the morning. This means a prolonged downtime between 6 pm and 9 am GMT on the next morning for both nights. Only the buildfarm will be affected, that means login.opencsw.org buildfarm.opencsw.org from the outside and all hosts reachable on the internal network from there. Sorry for the inconvenience -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From maciej at opencsw.org Thu Jul 18 17:05:23 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Thu, 18 Jul 2013 16:05:23 +0100 Subject: [csw-maintainers] Python 2.7 Message-ID: Peter Felecan contacted me some time ago about building Python 2.7 and possibly transitioning from 2.6 to 2.7. Here's r21514 which has been recently applied to the 2.7 build branch: http://lists.opencsw.org/pipermail/devel/2013-July/026807.html It takes the set of patches from 2.6 and applies them to 2.7. Many of the patches applied to the 2.6 are one of the following: - they're trying to fix a problem which is already fixed 2.7 upstream - they're trying to fix a problem that never was a problem - they're trying to fix a problem that does not affect Solaris 10 - they're actively breaking Python, such as the infamous "site" patch[1] The "site.diff" patch is horrible, should have never been allowed, and should never ever happen again. Out Python is currently broken[2] and it's very difficult for us to fix it because of the number of modules built into the unversioned directory. I propose that r21514 is rolled back. When that's done, let's discuss what do we want to do about Python 2.7. Peter's concern is that if we keep the versioned directory, it will be hard to transition to 2.7. Peter's objective is to package calibre, which needs Python 2.7 and a number of modules. I suggest that we build Python 2.7 as it was before r21514, in a versioned directory, and we build the necessary modules as CSWpy27-foo. Maybe we could patch Python 2.7 to look into the unversioned directory for modules ? as a backup solution. But we would need to see if this would break anything or not. Thoughts? Maciej [1] http://lists.opencsw.org/pipermail/devel/2009-May/009173.html [2] http://lists.opencsw.org/pipermail/users/2012-October/009338.html From pfelecan at opencsw.org Thu Jul 18 17:32:00 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 18 Jul 2013 17:32:00 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Thu, 18 Jul 2013 16:05:23 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > I propose that r21514 is rolled back. I committed a new version where there is no patch; it builds and packages alright; I'm focusing on 32 bit in the begining to avoid a long turn-around. > When that's done, let's discuss what do we want to do about Python > 2.7. Peter's concern is that if we keep the versioned directory, it > will be hard to transition to 2.7. Peter's objective is to package > calibre, which needs Python 2.7 and a number of modules. I suggest > that we build Python 2.7 as it was before r21514, in a versioned > directory, and we build the necessary modules as CSWpy27-foo. Naming the new packages CSWpy27-xxx is not a good idea from my standpoint. What I propose is when we rebuild a Python module package we just use the previous name and have a dependency on CSWpython27. Also, we should release a new 2.6 package where the /opt/csw/bin/python path doesn't exist anymore such as the default Python interpreter will be 2.7. > Maybe we could patch Python 2.7 to look into the unversioned directory > for modules ? as a backup solution. But we would need to see if this > would break anything or not. It breaks the building of Python 2.7 itself as it will mess with shared objects provided by Python 2.6 which are stored in the unversioned tree. If there is a way to make Python's byzantine build system to not look for shared objects in that path then everything is fine. -- Peter From maciej at opencsw.org Thu Jul 18 18:06:20 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Thu, 18 Jul 2013 17:06:20 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/18 Peter FELECAN : > "Maciej (Matchek) Blizi?ski" writes: >> When that's done, let's discuss what do we want to do about Python >> 2.7. Peter's concern is that if we keep the versioned directory, it >> will be hard to transition to 2.7. Peter's objective is to package >> calibre, which needs Python 2.7 and a number of modules. I suggest >> that we build Python 2.7 as it was before r21514, in a versioned >> directory, and we build the necessary modules as CSWpy27-foo. > > Naming the new packages CSWpy27-xxx is not a good idea from my > standpoint. What I propose is when we rebuild a Python module package we > just use the previous name and have a dependency on CSWpython27. This will break the 2.6 world. In your scenario: Before: CSWpython (which is Python 2.6) CSWpy-foo (built into the unversioned directory) #!/opt/csw/bin/python2.6 import foo # works After: CSWpython (which is Python 2.6) CSWpython27 CSWpy-foo (built into the 2.7 versioned directory) #!/opt/csw/bin/python2.6 import foo # fails > Also, we should release a new 2.6 package where the /opt/csw/bin/python > path doesn't exist anymore such as the default Python interpreter will > be 2.7. /opt/csw/bin/python should be a symlink driven by alternatives. We need to think what should be the default if both 2.6 and 2.7 Pythons are installed. I'm thinking that 2.6 should be the default for now. >> Maybe we could patch Python 2.7 to look into the unversioned directory >> for modules ? as a backup solution. But we would need to see if this >> would break anything or not. > > It breaks the building of Python 2.7 itself as it will mess with shared > objects provided by Python 2.6 which are stored in the unversioned > tree. If there is a way to make Python's byzantine build system to not > look for shared objects in that path then everything is fine. So why not simply let Python do its thing, where 2.6 is isolated from 2.7? This is the way it works out of the box. You just build stuff you need for 2.7 a and you're done. Maciej From bwalton at opencsw.org Thu Jul 18 19:30:57 2013 From: bwalton at opencsw.org (Ben Walton) Date: Thu, 18 Jul 2013 18:30:57 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: I spent a good amount of effort last fall on 2.7 packaging. I have working packages and rebuilt a pile of modules... I just didn't get all the way through the stack. There is a wiki page tracking the process. This could be dusted off with little effort. The module directory needs to be versions. Madness lies in the other direction. :) Thanks -Ben On Jul 18, 2013 5:07 PM, "Maciej (Matchek) Blizi?ski" wrote: > 2013/7/18 Peter FELECAN : > > "Maciej (Matchek) Blizi?ski" writes: > >> When that's done, let's discuss what do we want to do about Python > >> 2.7. Peter's concern is that if we keep the versioned directory, it > >> will be hard to transition to 2.7. Peter's objective is to package > >> calibre, which needs Python 2.7 and a number of modules. I suggest > >> that we build Python 2.7 as it was before r21514, in a versioned > >> directory, and we build the necessary modules as CSWpy27-foo. > > > > Naming the new packages CSWpy27-xxx is not a good idea from my > > standpoint. What I propose is when we rebuild a Python module package we > > just use the previous name and have a dependency on CSWpython27. > > This will break the 2.6 world. In your scenario: > > Before: > CSWpython (which is Python 2.6) > CSWpy-foo (built into the unversioned directory) > > #!/opt/csw/bin/python2.6 > import foo # works > > After: > CSWpython (which is Python 2.6) > CSWpython27 > CSWpy-foo (built into the 2.7 versioned directory) > > #!/opt/csw/bin/python2.6 > import foo # fails > > > Also, we should release a new 2.6 package where the /opt/csw/bin/python > > path doesn't exist anymore such as the default Python interpreter will > > be 2.7. > > /opt/csw/bin/python should be a symlink driven by alternatives. We > need to think what should be the default if both 2.6 and 2.7 Pythons > are installed. I'm thinking that 2.6 should be the default for now. > > >> Maybe we could patch Python 2.7 to look into the unversioned directory > >> for modules - as a backup solution. But we would need to see if this > >> would break anything or not. > > > > It breaks the building of Python 2.7 itself as it will mess with shared > > objects provided by Python 2.6 which are stored in the unversioned > > tree. If there is a way to make Python's byzantine build system to not > > look for shared objects in that path then everything is fine. > > So why not simply let Python do its thing, where 2.6 is isolated from > 2.7? This is the way it works out of the box. You just build stuff you > need for 2.7 a and you're done. > > Maciej > _______________________________________________ > maintainers mailing list > maintainers at lists.opencsw.org > https://lists.opencsw.org/mailman/listinfo/maintainers > .:: This mailing list's archive is public. ::. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maciej at opencsw.org Fri Jul 19 08:49:02 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 19 Jul 2013 07:49:02 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/18 Ben Walton : > I spent a good amount of effort last fall on 2.7 packaging. I have working > packages and rebuilt a pile of modules... I just didn't get all the way > through the stack. There is a wiki page tracking the process. http://wiki.opencsw.org/project-python2libdir The writeup is about Python 2.6, looks like it was a 2.6-related work. We also have Python 3.3: http://www.opencsw.org/packages/CSWpython33/ We do not have any modules built for 3.3. Maciej From jh at opencsw.org Fri Jul 19 09:19:15 2013 From: jh at opencsw.org (Jan Holzhueter) Date: Fri, 19 Jul 2013 09:19:15 +0200 Subject: [csw-maintainers] Downtime of farm on 18. and 22. July In-Reply-To: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> References: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> Message-ID: <51E8E873.7060607@opencsw.org> Hi, the power is back up, but I have a problem to get the builfarm server back online. It's hanging and I have no clue what it is at the moment Sorry for that. Greetings Jan Am 18.07.13 10:27, schrieb Dagobert Michelsen: > Hi folks, > > our power company will have a 2 hour power shutdown on the nights between 18. and 19. July > and between 22. and 23. July. As our UPS doesn't stand this long we are shutting down the > machines in the evening and boot everything up in the morning. This means a prolonged downtime > between 6 pm and 9 am GMT on the next morning for both nights. Only the buildfarm will be > affected, that means > login.opencsw.org > buildfarm.opencsw.org > from the outside and all hosts reachable on the internal network from there. > > > Sorry for the inconvenience > > -- Dago > From maciej at opencsw.org Fri Jul 19 10:04:16 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 19 Jul 2013 09:04:16 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: Here's how I propose we build modules for different Python versions: >From d905994a8496be4c1c3daea2df53df320e100e31 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Maciej=20Blizi=C5=84ski?= Date: Fri, 19 Jul 2013 08:50:45 +0100 Subject: [PATCH] Build Python modules for different versions --- categories/python/category.mk | 6 +++--- gar.conf.mk | 28 ++++++++++++++++++++++++++++ gar.lib.mk | 6 +++--- 3 files changed, 34 insertions(+), 6 deletions(-) diff --git a/categories/python/category.mk b/categories/python/category.mk index 5c69f0f..daaf10d 100644 --- a/categories/python/category.mk +++ b/categories/python/category.mk @@ -1,5 +1,5 @@ # Add a dependency to CSWpython -_EXTRA_GAR_PKGS += CSWpython +_EXTRA_GAR_PKGS += $(PYTHON_PACKAGE) # For the record, do not include the following line: # _MERGE_EXCLUDE_CATEGORY += .*\.egg-info.* @@ -28,11 +28,11 @@ TEST_TARGET ?= test LICENSE ?= PKG-INFO SPKG_SOURCEURL ?= http://pypi.python.org/pypi/$(NAME) MASTER_SITES ?= $(PYPI_MIRROR) -PACKAGES ?= CSWpy-$(DASHED_NAME) +PACKAGES ?= $(PYTHON_MODULE_PACKAGE_PREFIX)$(DASHED_NAME) # for use in any references by specific recipes so it can be replaced easily # across the tree. this could later be parameterized for use by multiple # versions of python too. -SITE_PACKAGES = $(libdir)/python2.6/site-packages +SITE_PACKAGES = $(PYTHON_SITE_PACKAGES) include gar/gar.mk diff --git a/gar.conf.mk b/gar.conf.mk index 433faa2..eeb0a81 100644 --- a/gar.conf.mk +++ b/gar.conf.mk @@ -854,6 +854,34 @@ endif INSTALL_SCRIPTS ?= $(WORKSRC)/Makefile +################################################################################ +# +# Python configuration + +PYTHON_VERSION ?= 2_6 + +PYTHON_EXECUTABLE_2_6 = /opt/csw/bin/python2.6 +PYTHON_EXECUTABLE_2_7 = /opt/csw/bin/python2.7 +PYTHON_EXECUTABLE_3_3 = /opt/csw/bin/python3.3 +PYTHON_EXECUTABLE = $(PYTHON_EXECUTABLE_$(PYTHON_VERSION)) + +PYTHON_PACKAGE_2_6 = CSWpython +PYTHON_PACKAGE_2_7 = CSWpython27 +PYTHON_PACKAGE_3_3 = CSWpython33 +PYTHON_PACKAGE = $(PYTHON_PACKAGE_$(PYTHON_VERSION)) + +PYTHON_SITE_PACKAGES_2_6 = $(libdir)/python/site-packages +PYTHON_SITE_PACKAGES_2_7 = $(libdir)/python2.7/site-packages +PYTHON_SITE_PACKAGES_3_3 = $(libdir)/python3.3/site-packages +PYTHON_SITE_PACKAGES = $(PYTHON_SITE_PACKAGES_$(PYTHON_VERSION)) + +PYTHON_MODULE_PACKAGE_PREFIX_2_6 = CSWpy- +PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy27- +PYTHON_MODULE_PACKAGE_PREFIX_3_3 = CSWpy33- +PYTHON_MODULE_PACKAGE_PREFIX = $(PYTHON_MODULE_PACKAGE_PREFIX_$(PYTHON_VERSION)) + +################################################################################ + # Global environment export PATH PKG_CONFIG_PATH diff --git a/gar.lib.mk b/gar.lib.mk index 781bd5d..5f00970 100644 --- a/gar.lib.mk +++ b/gar.lib.mk @@ -905,7 +905,7 @@ build-%/waf: PYBUILD_CMD ?= build build-%/setup.py: @echo " ==> Running setup.py $(PYBUILD_TYPE) in $*" - @( cd $* ; $(BUILD_ENV) python ./setup.py $(PYBUILD_CMD) $(BUILD_ARGS) ) + @( cd $* ; $(BUILD_ENV) $(PYTHON_EXECUTABLE) ./setup.py $(PYBUILD_CMD) $(BUILD_ARGS) ) @$(MAKECOOKIE) #################### CLEAN RULES #################### @@ -982,7 +982,7 @@ test-%/waf: test-%/setup.py: @echo " ==> Running setup.py test in $*" - @( cd $* ; cd $(OBJDIR) ; $(TEST_ENV) python ./setup.py test $(TEST_ARGS) ) + @( cd $* ; cd $(OBJDIR) ; $(TEST_ENV) $(PYTHON_EXECUTABLE) ./setup.py test $(TEST_ARGS) ) @$(MAKECOOKIE) ################# INSTALL RULES #################### @@ -1030,7 +1030,7 @@ install-%/waf: PYINSTALL_CMD ?= install install-%/setup.py: @echo " ==> Running setup.py $(PYINSTALL_CMD) in $*" - @( cd $* ; $(INSTALL_ENV) python ./setup.py $(PYINSTALL_CMD) $(INSTALL_ARGS) ) + @( cd $* ; $(INSTALL_ENV) $(PYTHON_EXECUTABLE) ./setup.py $(PYINSTALL_CMD) $(INSTALL_ARGS) ) @$(MAKECOOKIE) # pkg-config scripts -- 1.8.3.1 From jh at opencsw.org Fri Jul 19 10:34:14 2013 From: jh at opencsw.org (Jan Holzhueter) Date: Fri, 19 Jul 2013 10:34:14 +0200 Subject: [csw-maintainers] Downtime of farm on 18. and 22. July In-Reply-To: <51E8E873.7060607@opencsw.org> References: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> <51E8E873.7060607@opencsw.org> Message-ID: <51E8FA06.60603@opencsw.org> Hi, ok we are back up. the rpool was 96% full which caused zfs to go into the slow motion state :) Hope after I removed a few shapshots that it's now faster. I only bootet the main zone and systems. (unstable9/10/11) mysql and cswsign. If anyone needs a special system please let me know and I will boot it. @everyone with the cswsign key. Please start the daemon Greetings Jan Am 19.07.13 09:19, schrieb Jan Holzhueter: > Hi, > the power is back up, > but I have a problem to get the builfarm server back online. > It's hanging and I have no clue what it is at the moment > Sorry for that. > > Greetings > Jan > > > Am 18.07.13 10:27, schrieb Dagobert Michelsen: >> Hi folks, >> >> our power company will have a 2 hour power shutdown on the nights between 18. and 19. July >> and between 22. and 23. July. As our UPS doesn't stand this long we are shutting down the >> machines in the evening and boot everything up in the morning. This means a prolonged downtime >> between 6 pm and 9 am GMT on the next morning for both nights. Only the buildfarm will be >> affected, that means >> login.opencsw.org >> buildfarm.opencsw.org >> from the outside and all hosts reachable on the internal network from there. >> >> >> Sorry for the inconvenience >> >> -- Dago >> > From pfelecan at opencsw.org Fri Jul 19 10:42:03 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 19 Jul 2013 10:42:03 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Fri, 19 Jul 2013 09:04:16 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > +PYTHON_MODULE_PACKAGE_PREFIX_2_6 = CSWpy- > +PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy27- > +PYTHON_MODULE_PACKAGE_PREFIX_3_3 = CSWpy33- > +PYTHON_MODULE_PACKAGE_PREFIX = > $(PYTHON_MODULE_PACKAGE_PREFIX_$(PYTHON_VERSION)) I think that using a different prefix for 2.6 and 2.7 is not a good idea: it will hinder the replacement of the current Python, 2.6, by the new one, 2.7. Lets use the same prefix for 2.x to smooth the transition. IMHO the objective is to make 2.7 the current 2.x Python. We don't want to support a CSWpy-foo and CSWpy27-foo, isn't it? What's the rationale of that? Anyhow, I don't get what's the issue with a 2.6 package using the 2.7 interpreter. The only one that I see is if there is a binary component tightly linked to a give version. Which are those "binary" packages anyway? Using a different prefix for 3.x is a good thing. -- Peter From pfelecan at opencsw.org Fri Jul 19 11:04:52 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 19 Jul 2013 11:04:52 +0200 Subject: [csw-maintainers] Python 2.7 References: Message-ID: Here is what I've done: - created a patch to append /opt/csw/lib/python/site-packages at the end of the search path for modules - rebuilt to verify that there is no failure when 2.6 is installed on the build system - installed 2.7 in parallel with 2.6; there is a conflict on /opt/csw/bin/python symbolic link but for the moment I ignore this - packaging Calibre, which uses construct only available in 2.7, such as set literals, dictionary comprehensions, is now possible. -- Peter From maciej at opencsw.org Fri Jul 19 17:26:47 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 19 Jul 2013 16:26:47 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/19 Peter FELECAN : > Here is what I've done: > > - created a patch to append /opt/csw/lib/python/site-packages at the end > of the search path for modules Does it work? When I've suggested this solution, you said it wouldn't. If it does, cool! > - rebuilt to verify that there is no failure when 2.6 is installed on the > build system > - installed 2.7 in parallel with 2.6; there is a conflict on > /opt/csw/bin/python symbolic link but for the moment I ignore this 2.6 needs to be rebuilt with alternatives. When you're at it, could you do it as well? > - packaging Calibre, which uses construct only available in 2.7, such as > set literals, dictionary comprehensions, is now possible. Nice. I gave a little more thought to the CSWpy27- prefix and I think it's impossible to avoid it if we ever want to move away from the horrible unversioned directory. If we start building modules with the CSWpy27- prefix, then at least we have a chance of moving away from the unversioned directory and fixing our 2.x Python. Maciej From pfelecan at opencsw.org Fri Jul 19 19:29:48 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 19 Jul 2013 19:29:48 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Fri, 19 Jul 2013 16:26:47 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/19 Peter FELECAN : >> Here is what I've done: >> >> - created a patch to append /opt/csw/lib/python/site-packages at the end >> of the search path for modules > > Does it work? When I've suggested this solution, you said it wouldn't. > If it does, cool! This is what I experienced when I forced an unversioned tree for 2.7: I got issues building. With a versioned tree there is no issue and I can understand why, following the search algorithm. >> - rebuilt to verify that there is no failure when 2.6 is installed on the >> build system >> - installed 2.7 in parallel with 2.6; there is a conflict on >> /opt/csw/bin/python symbolic link but for the moment I ignore this > > 2.6 needs to be rebuilt with alternatives. When you're at it, could > you do it as well? I'm not sure yet as I don't think that we should use 2.6 alternatively with 2.7. I'm still hoping that someone explains me why code written for 2.6 cannot be run by a 2.7 interpreter. >> - packaging Calibre, which uses construct only available in 2.7, such as >> set literals, dictionary comprehensions, is now possible. > > Nice. > > I gave a little more thought to the CSWpy27- prefix and I think it's > impossible to avoid it if we ever want to move away from the horrible > unversioned directory. If we start building modules with the CSWpy27- > prefix, then at least we have a chance of moving away from the > unversioned directory and fixing our 2.x Python. I tend to disagree. What I have in mind is this: 1. 2.7 has /opt/csw/lib/pyhton/site-packages as the last item in sys.path; thus packages in it can be executed without issues if they are not of "binary" type (still needs a definition for "binary" package) 2. 2.6 is repackaged: - to remove the /opt/csw/bin/python and /opt/csw/share/man/man1/python.1 to avoid conflicts with 2.7; this will insure that "binary" modules can be run and those using the explicit shebang toward python2.6 are not lost. 3. 2.7 becomes the default python for our distribution, thus new packages depends on it and the gar python recipe adapts to this but using CSWpy- prefix; I'm hesitating to add that we can obsolete CSWpython-xxx by CSWpython27-xxx 4. when version bumps are committed for existing packages, the transition is implicit 5. when there is an issue with an existing package, built using 2.6 with unversioned tree, we re-spin it; this is why we have Mantis isn't it; I think that a Python working group can be created to take into account packages for retired maintainers. What do you think? -- Peter From maciej at opencsw.org Fri Jul 19 19:38:42 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 19 Jul 2013 18:38:42 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/19 Peter FELECAN > (snip) > 3. 2.7 becomes the default python for our distribution, thus new > packages depends on it and the gar python recipe adapts to this but > using CSWpy- prefix; I'm hesitating to add that we can obsolete > CSWpython-xxx by CSWpython27-xxx I can imagine the following scenario: User: I upgraded CSW stuff and my application is not working, it fails with "ImportError: No module named foo" Us: Which Python are you using? User: Python 2.6 Us: We made Python 2.7 the default and we're removing Python 2.6 support. User: But my application requires Python 2.6. Us: Then update your application. User: It's a third party application, we cannot modify it. > 4. when version bumps are committed for existing packages, the transition > is implicit > > 5. when there is an issue with an existing package, built using 2.6 with > unversioned tree, we re-spin it; this is why we have Mantis isn't it; > I think that a Python working group can be created to take into > account packages for retired maintainers. > > What do you think? We might make Python 2.7 the default but it doesn't mean that our users will. I vote for not breaking Python 2.6 for our users. Maciej From pfelecan at opencsw.org Sat Jul 20 09:28:31 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Sat, 20 Jul 2013 09:28:31 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Fri, 19 Jul 2013 18:38:42 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/19 Peter FELECAN >> (snip) >> 3. 2.7 becomes the default python for our distribution, thus new >> packages depends on it and the gar python recipe adapts to this but >> using CSWpy- prefix; I'm hesitating to add that we can obsolete >> CSWpython-xxx by CSWpython27-xxx > > I can imagine the following scenario: > > User: I upgraded CSW stuff and my application is not working, it fails > with "ImportError: No module named foo" > Us: Which Python are you using? > User: Python 2.6 > Us: We made Python 2.7 the default and we're removing Python 2.6 support. > User: But my application requires Python 2.6. > Us: Then update your application. > User: It's a third party application, we cannot modify it. What about this: > 2013/7/19 Peter FELECAN >> I'm still hoping that someone explains me why code written for 2.6 >> cannot be run by a 2.7 interpreter. which is, I think, crucial to our discussion. -- Peter From bwalton at opencsw.org Sat Jul 20 09:31:38 2013 From: bwalton at opencsw.org (Ben Walton) Date: Sat, 20 Jul 2013 08:31:38 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: On Fri, Jul 19, 2013 at 7:49 AM, Maciej (Matchek) Blizi?ski wrote: > 2013/7/18 Ben Walton : >> I spent a good amount of effort last fall on 2.7 packaging. I have working >> packages and rebuilt a pile of modules... I just didn't get all the way >> through the stack. There is a wiki page tracking the process. > > http://wiki.opencsw.org/project-python2libdir > > The writeup is about Python 2.6, looks like it was a 2.6-related work. If I were testifying in court, I'd have sworn it was 2.7 I worked on. But it seems I'd be a poor witness. :( If the plan is to get to 2.7 (which I think is the right thing) then maybe we could take what I've done, which puts libraries where they belong and then build all of the new modules for 2.7? Thanks -Ben From bwalton at opencsw.org Sat Jul 20 09:34:41 2013 From: bwalton at opencsw.org (Ben Walton) Date: Sat, 20 Jul 2013 08:34:41 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: On Sat, Jul 20, 2013 at 8:28 AM, Peter FELECAN wrote: > "Maciej (Matchek) Blizi?ski" writes: > >> 2013/7/19 Peter FELECAN >>> (snip) >>> 3. 2.7 becomes the default python for our distribution, thus new >>> packages depends on it and the gar python recipe adapts to this but >>> using CSWpy- prefix; I'm hesitating to add that we can obsolete >>> CSWpython-xxx by CSWpython27-xxx >> >> I can imagine the following scenario: >> >> User: I upgraded CSW stuff and my application is not working, it fails >> with "ImportError: No module named foo" >> Us: Which Python are you using? >> User: Python 2.6 >> Us: We made Python 2.7 the default and we're removing Python 2.6 support. >> User: But my application requires Python 2.6. >> Us: Then update your application. >> User: It's a third party application, we cannot modify it. > > What about this: I'd be very tempted to release the 2.6 stack with proper library directories and fill in missing modules as needed. The preference would be to build modules for 2.7 but we could reasonably easily build for both... > >> 2013/7/19 Peter FELECAN > >>> I'm still hoping that someone explains me why code written for 2.6 >>> cannot be run by a 2.7 interpreter. > > which is, I think, crucial to our discussion. Most of the modules would work. Binary stuff doesn't (or at least may not) as you've mentioned. Versioned library paths are the way to go though. Not fixing this now just kicks the can down the road. Thanks -Ben From pfelecan at opencsw.org Sat Jul 20 09:59:10 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Sat, 20 Jul 2013 09:59:10 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: (Ben Walton's message of "Sat, 20 Jul 2013 08:34:41 +0100") References: Message-ID: Ben Walton writes: > On Sat, Jul 20, 2013 at 8:28 AM, Peter FELECAN wrote: >> "Maciej (Matchek) Blizi?ski" writes: >> >>> 2013/7/19 Peter FELECAN >>>> (snip) >>>> 3. 2.7 becomes the default python for our distribution, thus new >>>> packages depends on it and the gar python recipe adapts to this but >>>> using CSWpy- prefix; I'm hesitating to add that we can obsolete >>>> CSWpython-xxx by CSWpython27-xxx >>> >>> I can imagine the following scenario: >>> >>> User: I upgraded CSW stuff and my application is not working, it fails >>> with "ImportError: No module named foo" >>> Us: Which Python are you using? >>> User: Python 2.6 >>> Us: We made Python 2.7 the default and we're removing Python 2.6 support. >>> User: But my application requires Python 2.6. >>> Us: Then update your application. >>> User: It's a third party application, we cannot modify it. >> >> What about this: > > I'd be very tempted to release the 2.6 stack with proper library > directories and fill in missing modules as needed. The preference > would be to build modules for 2.7 but we could reasonably easily build > for both... The idea is to make the transition to 2.7. Additional work on 2.6 is a effort expense that we cannot afford given our resources. Concentrating on 2.7 and using the least effort path is crucial. >>> 2013/7/19 Peter FELECAN >> >>>> I'm still hoping that someone explains me why code written for 2.6 >>>> cannot be run by a 2.7 interpreter. >> >> which is, I think, crucial to our discussion. > > > Most of the modules would work. Binary stuff doesn't (or at least may > not) as you've mentioned. Versioned library paths are the way to go > though. Not fixing this now just kicks the can down the road. If "Most" means 90% then what I proposed is a good compromise. If we can easily identify which are the "binary" modules, we have a perfect solution with a minimal investment. Look at what Debian has done to make the transition. They are not using specific 2.7 packages, the differentiation is 2 versus 3 which is understandable as there is an incompatibility between code written for 2 running in 3. I would like to be understood that what I'm aiming at is 2 pronged: 1. minimal investment 2. fastest turn around -- Peter From maciej at opencsw.org Sat Jul 20 14:13:26 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sat, 20 Jul 2013 13:13:26 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/20 Peter FELECAN > > Look at what Debian has done to make the transition. They are not using > specific 2.7 packages, the differentiation is 2 versus 3 which is > understandable as there is an incompatibility between code written for 2 > running in 3. I don't think they broke Python 2.6. Or did they? They have a special mechanism for sharing modules between 2.6 and 2.7. > I would like to be understood that what I'm aiming at is 2 pronged: > > 1. minimal investment > 2. fastest turn around With Python 2.6 being expendable? Maciej From pfelecan at opencsw.org Sat Jul 20 15:05:04 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Sat, 20 Jul 2013 15:05:04 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Sat, 20 Jul 2013 13:13:26 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/20 Peter FELECAN >> >> Look at what Debian has done to make the transition. They are not using >> specific 2.7 packages, the differentiation is 2 versus 3 which is >> understandable as there is an incompatibility between code written for 2 >> running in 3. > > I don't think they broke Python 2.6. Or did they? They have a special > mechanism for sharing modules between 2.6 and 2.7. Which is? Anyhow, they don't have the equivalent of CSWpy27-foo. A 2.x package is a 2.x package. >> I would like to be understood that what I'm aiming at is 2 pronged: >> >> 1. minimal investment >> 2. fastest turn around > > With Python 2.6 being expendable? Why do you think so? In my current endeavor, I come to realize that our Python eco-system is brain damaged. What I propose is not to lobotomize, but to have brain surgery. -- Peter From maciej at opencsw.org Sat Jul 20 18:16:08 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sat, 20 Jul 2013 17:16:08 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/20 Peter FELECAN : > "Maciej (Matchek) Blizi?ski" writes: > >> 2013/7/20 Peter FELECAN >>> >>> Look at what Debian has done to make the transition. They are not using >>> specific 2.7 packages, the differentiation is 2 versus 3 which is >>> understandable as there is an incompatibility between code written for 2 >>> running in 3. >> >> I don't think they broke Python 2.6. Or did they? They have a special >> mechanism for sharing modules between 2.6 and 2.7. > > Which is? Anyhow, they don't have the equivalent of CSWpy27-foo. A 2.x > package is a 2.x package. The Debian policy discusses 2.5 vs 2.6, but it's the same discussion. http://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html "A common location to share, across Python versions, arch-independent files which would otherwise go to the directory of system public modules is /usr/share/pyshared." >>> I would like to be understood that what I'm aiming at is 2 pronged: >>> >>> 1. minimal investment >>> 2. fastest turn around >> >> With Python 2.6 being expendable? > > Why do you think so? > > In my current endeavor, I come to realize that our Python eco-system is > brain damaged. > > What I propose is not to lobotomize, but to have brain surgery. A plan that could work, is the following: - rebuild 2.6 with /opt/csw/bin/python controlled by the alternatives system - build 2.7 as you did, with /opt/csw/lib/python/site-packages as an extra module search path, and with /opt/csw/bin/python also controlled by alternatives - keep using the CSWpy- package name prefix - keep building 2.x modules with Python 2.6 only; never build modules with 2.7 and never put them into the /opt/csw/lib/python2.7/site-packages directory - keep Python 3.x completely separate, with CSWpy33- package name prefix The downside is that we're still using the bad site packages directory. The upside is that this scenario meets all the basic criteria: - keep 2.6 working - easy to do Maciej From pfelecan at opencsw.org Sat Jul 20 19:16:43 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Sat, 20 Jul 2013 19:16:43 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Sat, 20 Jul 2013 17:16:08 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/20 Peter FELECAN : >> "Maciej (Matchek) Blizi?ski" writes: >> >>> 2013/7/20 Peter FELECAN >>>> >>>> Look at what Debian has done to make the transition. They are not using >>>> specific 2.7 packages, the differentiation is 2 versus 3 which is >>>> understandable as there is an incompatibility between code written for 2 >>>> running in 3. >>> >>> I don't think they broke Python 2.6. Or did they? They have a special >>> mechanism for sharing modules between 2.6 and 2.7. >> >> Which is? Anyhow, they don't have the equivalent of CSWpy27-foo. A 2.x >> package is a 2.x package. > > The Debian policy discusses 2.5 vs 2.6, but it's the same discussion. > > http://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html > > "A common location to share, across Python versions, arch-independent > files which would otherwise go to the directory of system public > modules is /usr/share/pyshared." We can introduce this kind of location if we wish; I'm not being so warm toward that but why not. >>>> I would like to be understood that what I'm aiming at is 2 pronged: >>>> >>>> 1. minimal investment >>>> 2. fastest turn around >>> >>> With Python 2.6 being expendable? >> >> Why do you think so? >> >> In my current endeavor, I come to realize that our Python eco-system is >> brain damaged. >> >> What I propose is not to lobotomize, but to have brain surgery. > > A plan that could work, is the following: > > - rebuild 2.6 with /opt/csw/bin/python controlled by the alternatives system > - build 2.7 as you did, with /opt/csw/lib/python/site-packages as an > extra module search path, and with /opt/csw/bin/python also controlled > by alternatives Why alternatives when we wish to make a transition from 2.6 to 2.7? In my proposition we would have python -> python2 -> python2.7 and python2.6 is there for those really needing it, which can put in their scripts #! /opt/csw/bin/python2.6 > - keep using the CSWpy- package name prefix Alright > - keep building 2.x modules with Python 2.6 only; never build modules > with 2.7 and never put them into the > /opt/csw/lib/python2.7/site-packages directory Why's that? This precludes a transition. > - keep Python 3.x completely separate, with CSWpy33- package name prefix I agree. > The downside is that we're still using the bad site packages > directory. The upside is that this scenario meets all the basic > criteria: > - keep 2.6 working Why is that necessary in the long term? Anyhow, I don't get why we cannot do this in the unstable catalog. Support for 2.7 and associated eco-system should be a transition criteria from unstable to the next to next stable / named catalog. > - easy to do Sure. The easiest would be to do nothing. -- Peter From maciej at opencsw.org Sun Jul 21 02:13:41 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sun, 21 Jul 2013 01:13:41 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/20 Peter FELECAN : >> The Debian policy discusses 2.5 vs 2.6, but it's the same discussion. >> >> http://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html >> >> "A common location to share, across Python versions, arch-independent >> files which would otherwise go to the directory of system public >> modules is /usr/share/pyshared." > > We can introduce this kind of location if we wish; I'm not being so warm > toward that but why not. I'm not saying that this is what we should do; but I'm saying that that's what Debian is doing to share modules between 2.x Python versions. >>>>> I would like to be understood that what I'm aiming at is 2 pronged: >>>>> >>>>> 1. minimal investment >>>>> 2. fastest turn around >>>> >>>> With Python 2.6 being expendable? >>> >>> Why do you think so? >>> >>> In my current endeavor, I come to realize that our Python eco-system is >>> brain damaged. >>> >>> What I propose is not to lobotomize, but to have brain surgery. >> >> A plan that could work, is the following: >> >> - rebuild 2.6 with /opt/csw/bin/python controlled by the alternatives system >> - build 2.7 as you did, with /opt/csw/lib/python/site-packages as an >> extra module search path, and with /opt/csw/bin/python also controlled >> by alternatives > > Why alternatives when we wish to make a transition from 2.6 to 2.7? > > In my proposition we would have python -> python2 -> python2.7 and > python2.6 is there for those really needing it, which can put in their > scripts #! /opt/csw/bin/python2.6 > >> - keep using the CSWpy- package name prefix > > Alright > >> - keep building 2.x modules with Python 2.6 only; never build modules >> with 2.7 and never put them into the >> /opt/csw/lib/python2.7/site-packages directory > > Why's that? This precludes a transition. Does it? When you have python2.7 looking for modules in the /opt/csw/lib/python/site-packages (unversioned) directory, and you make /opt/csw/bin/python2.7 the higher priority alternative, you can install Python 2.6 and everything will work. You can install Python 2.7 and everything will work too. You've transitioned already, no? >> - keep 2.6 working > > Why is that necessary in the long term? Our users will migrate from 2.6 into 2.7 at a different point in time that we do. We must first provide a package catalog in which both 2.6 and 2.7 versions work. Then our users will take their time and update their installations, and their Python-dependent software. We can use our release cycle to deprecate Python 2.6, but there still must be an overlap. > Anyhow, I don't get why we > cannot do this in the unstable catalog. Support for 2.7 and associated > eco-system should be a transition criteria from unstable to the next to > next stable / named catalog. Unstable is unstable... but preferably not so unstable that Python doesn't work any more. If we want to perform a grand rebuild, we have the beanie catalog specifically for this kind of work. Maciej From dam at opencsw.org Sun Jul 21 14:12:20 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Sun, 21 Jul 2013 14:12:20 +0200 Subject: [csw-maintainers] GNU hackers meeting in Paris Message-ID: <9EF5E523-9E33-4A3B-89C4-BCDAC7F9B846@opencsw.org> Hi folks, I just noticed there is a GNU hackers meeting in Paris from 22. to 25. August: http://www.gnu.org/ghm/2013/paris/ Unfortunately I can't come... Maybe this is interesting for some of our France (=Paris) maintainers :-) Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From pfelecan at opencsw.org Mon Jul 22 10:47:28 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 22 Jul 2013 10:47:28 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Sun, 21 Jul 2013 01:13:41 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/20 Peter FELECAN : >> Anyhow, I don't get why we >> cannot do this in the unstable catalog. Support for 2.7 and associated >> eco-system should be a transition criteria from unstable to the next to >> next stable / named catalog. > > Unstable is unstable... but preferably not so unstable that Python > doesn't work any more. If we want to perform a grand rebuild, we have > the beanie catalog specifically for this kind of work. Again, I don't get why we cannot do this kind of transition in unstable. Maybe I do not understand what we call unstable. Especially when 2.6 is not doomed by what I propose. Also, you don't state the rationale about why a 2.6 code doesn't work in a 2.7 interpreter (except "binary" modules for which I don't have a clear identification yet, neither in quality nor in quantity). -- Peter From maciej at opencsw.org Mon Jul 22 12:36:13 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 22 Jul 2013 11:36:13 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: Just a quick update: I did some testing and it seems like 2.6 with a versioned directory plus an extra path in the module search path does work. It's now pushed into unstable ? please test. Maciej -------------- next part -------------- An HTML attachment was scrubbed... URL: From pfelecan at opencsw.org Mon Jul 22 13:59:21 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 22 Jul 2013 13:59:21 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 22 Jul 2013 11:36:13 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > Just a quick update: I did some testing and it seems like 2.6 with a > versioned directory plus an extra path in the module search path does work. > It's now pushed into unstable ? please test. I'll do. Do you mind committing the patch that you proposed in [1] with the exception of different naming schemes for 2.7, i.e. no CSWpy27- prefix? What about CSWpython27 and co: do yo release them? [1] http://lists.opencsw.org/pipermail/maintainers/2013-July/018239.html -- Peter From maciej at opencsw.org Mon Jul 22 15:46:52 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 22 Jul 2013 14:46:52 +0100 Subject: [csw-maintainers] Downtime of farm on 18. and 22. July In-Reply-To: <51E8FA06.60603@opencsw.org> References: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> <51E8E873.7060607@opencsw.org> <51E8FA06.60603@opencsw.org> Message-ID: 2013/7/19 Jan Holzhueter > > @everyone with the cswsign key. Please start the daemon Done. Sorry for the delay. From maciej at opencsw.org Mon Jul 22 18:04:25 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 22 Jul 2013 17:04:25 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/22 Peter FELECAN : > Do you mind committing the patch that you proposed in [1] with the > exception of different naming schemes for 2.7, i.e. no CSWpy27- prefix? The naming remains a problem, if we want to share modules between 2.6 and 2.7, we need to do something similar to what Debian did. > What about CSWpython27 and co: do yo release them? I've switched both Python versions to alternatives, so we're ready to release them. The main problem to solve is how do we share modules between 2.6 and 2.7. As to why 2.6 code wouldn't work in 2.7 ? Python 2.7 is meant to be backwards compatible, so there isn't an obvious example. In a perfect world you'd just flip the switch, but in reality you cannot migrate your production over to 2.7 without prior testing and a migration procedure. The migration procedure would include changing #!/opt/csw/bin/python2.6 to #!/opt/csw/bin/python2.7. In short, you cannot make our users run 2.7 on our call. They need to switch to 2.7 at their own schedule, and we need to make it very clear about when we're pulling the plug on 2.6. Maciej From jh at opencsw.org Tue Jul 23 07:49:11 2013 From: jh at opencsw.org (Jan Holzhueter) Date: Tue, 23 Jul 2013 07:49:11 +0200 Subject: [csw-maintainers] Downtime of farm on 18. and 22. July In-Reply-To: <51E8E873.7060607@opencsw.org> References: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> <51E8E873.7060607@opencsw.org> Message-ID: <51EE1957.8060402@opencsw.org> Hi, we are back up and running. I hope now for a longer time :) Someone with the sign key please enable it again. Greetings Jan From maciej at opencsw.org Tue Jul 23 09:39:30 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 23 Jul 2013 08:39:30 +0100 Subject: [csw-maintainers] Downtime of farm on 18. and 22. July In-Reply-To: <51EE1957.8060402@opencsw.org> References: <77AA5630-6DE8-47E8-9DAF-1EA4D1A00B40@opencsw.org> <51E8E873.7060607@opencsw.org> <51EE1957.8060402@opencsw.org> Message-ID: 2013/7/23 Jan Holzhueter > Someone with the sign key please enable it again. Done. From pfelecan at opencsw.org Tue Jul 23 09:43:49 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 23 Jul 2013 09:43:49 +0200 Subject: [csw-maintainers] Python 2.7 References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/22 Peter FELECAN : >> Do you mind committing the patch that you proposed in [1] with the >> exception of different naming schemes for 2.7, i.e. no CSWpy27- prefix? > > The naming remains a problem, if we want to share modules between 2.6 > and 2.7, we need to do something similar to what Debian did. Lets do it Debian way. Always a good reference, as you already mentioned: http://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html >> What about CSWpython27 and co: do yo release them? > > I've switched both Python versions to alternatives, so we're ready to > release them. The main problem to solve is how do we share modules > between 2.6 and 2.7. My question is: do you release 2.7? > As to why 2.6 code wouldn't work in 2.7 ? Python 2.7 is meant to be > backwards compatible, so there isn't an obvious example. In a perfect > world you'd just flip the switch, but in reality you cannot migrate > your production over to 2.7 without prior testing and a migration > procedure. The migration procedure would include changing > #!/opt/csw/bin/python2.6 to #!/opt/csw/bin/python2.7. >From my point of view this is not very convincing but you're the Python Czar consequently bowing to it. > In short, you cannot make our users run 2.7 on our call. They need to > switch to 2.7 at their own schedule, and we need to make it very clear > about when we're pulling the plug on 2.6. The moment of switch will be when this unstable becomes a named version, i.e. stable. However, nothing precludes to still have legacy 2.6 modules in it. -- Peter From dam at opencsw.org Tue Jul 23 18:19:20 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 23 Jul 2013 18:19:20 +0200 Subject: [csw-maintainers] Enlightenment WM anyone? Message-ID: <6D6BC450-D530-46EC-B466-C22E449F1649@opencsw.org> Hi Jake and others, I get constant warnings on the old CSWimlib m4 macros and would like to know if there is still interest in keeping (=updating) it. The only package using it is enlightenment which Jake packaged in 2004. I would be willing to fix CSWimlib if there is interest in updating enlightenment also, otherwise I suggest dropping both. http://www.opencsw.org/packages/imlib/ http://www.opencsw.org/packages/enlightenment/ Thoughts? Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From laurent at opencsw.org Tue Jul 23 22:09:55 2013 From: laurent at opencsw.org (Laurent Blume) Date: Tue, 23 Jul 2013 22:09:55 +0200 Subject: [csw-maintainers] Enlightenment WM anyone? In-Reply-To: <6D6BC450-D530-46EC-B466-C22E449F1649@opencsw.org> References: <6D6BC450-D530-46EC-B466-C22E449F1649@opencsw.org> Message-ID: <51EEE313.7070807@opencsw.org> Hello, I'm in favour of dropping. I can't see them as widely used applications, on any platform, they're niche. If somebody cares enough to pick them up, they'll be welcome to fix the warnings and maintain. Laurent On 23/07/2013 18:19, Dagobert Michelsen wrote: > Hi Jake and others, > > I get constant warnings on the old CSWimlib m4 macros and would like to know if there > is still interest in keeping (=updating) it. The only package using it is > enlightenment which Jake packaged in 2004. I would be willing to fix CSWimlib if > there is interest in updating enlightenment also, otherwise I suggest dropping both. > http://www.opencsw.org/packages/imlib/ > http://www.opencsw.org/packages/enlightenment/ > > Thoughts? > > > Best regards > > -- Dago > From bwalton at opencsw.org Tue Jul 23 22:17:37 2013 From: bwalton at opencsw.org (Ben Walton) Date: Tue, 23 Jul 2013 21:17:37 +0100 Subject: [csw-maintainers] Enlightenment WM anyone? In-Reply-To: <51EEE313.7070807@opencsw.org> References: <6D6BC450-D530-46EC-B466-C22E449F1649@opencsw.org> <51EEE313.7070807@opencsw.org> Message-ID: +1. On Jul 23, 2013 9:10 PM, "Laurent Blume" wrote: > Hello, > > I'm in favour of dropping. I can't see them as widely used applications, > on any platform, they're niche. If somebody cares enough to pick them up, > they'll be welcome to fix the warnings and maintain. > > Laurent > > On 23/07/2013 18:19, Dagobert Michelsen wrote: > >> Hi Jake and others, >> >> I get constant warnings on the old CSWimlib m4 macros and would like to >> know if there >> is still interest in keeping (=updating) it. The only package using it is >> enlightenment which Jake packaged in 2004. I would be willing to fix >> CSWimlib if >> there is interest in updating enlightenment also, otherwise I suggest >> dropping both. >> http://www.opencsw.org/**packages/imlib/ >> http://www.opencsw.org/**packages/enlightenment/ >> >> Thoughts? >> >> >> Best regards >> >> -- Dago >> >> > ______________________________**_________________ > maintainers mailing list > maintainers at lists.opencsw.org > https://lists.opencsw.org/**mailman/listinfo/maintainers > .:: This mailing list's archive is public. ::. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maciej at opencsw.org Thu Jul 25 09:27:05 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Thu, 25 Jul 2013 08:27:05 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/23 Peter FELECAN : > "Maciej (Matchek) Blizi?ski" writes: > >> 2013/7/22 Peter FELECAN : >>> Do you mind committing the patch that you proposed in [1] with the >>> exception of different naming schemes for 2.7, i.e. no CSWpy27- prefix? >> >> The naming remains a problem, if we want to share modules between 2.6 >> and 2.7, we need to do something similar to what Debian did. > > Lets do it Debian way. Always a good reference, as you already > mentioned: > http://www.debian.org/doc/packaging-manuals/python-policy/ch-python.html This document doesn't say too much about what they do to share python modules, it only mentions /usr/share/pyshared in one place without providing any details. I'll return to this later in the email. >>> What about CSWpython27 and co: do yo release them? >> >> I've switched both Python versions to alternatives, so we're ready to >> release them. The main problem to solve is how do we share modules >> between 2.6 and 2.7. > > My question is: do you release 2.7? I'm pushing 2.7 out to unstable right now. >> As to why 2.6 code wouldn't work in 2.7 ? Python 2.7 is meant to be >> backwards compatible, so there isn't an obvious example. In a perfect >> world you'd just flip the switch, but in reality you cannot migrate >> your production over to 2.7 without prior testing and a migration >> procedure. The migration procedure would include changing >> #!/opt/csw/bin/python2.6 to #!/opt/csw/bin/python2.7. > > From my point of view this is not very convincing but you're the Python > Czar consequently bowing to it. OK, here's a concrete example: $ python2.6 -c "range(5.0)" -c:1: DeprecationWarning: integer argument expected, got float $ echo $? 0 $ python2.7 -c "range(5.0)" Traceback (most recent call last): File "", line 1, in TypeError: range() integer end argument expected, got float. $ echo $? 1 I'm not denying that the backwards compatibility is high, I'm saying that if you run Python in production, you must test before switching and you must have the options of rolling back. >> In short, you cannot make our users run 2.7 on our call. They need to >> switch to 2.7 at their own schedule, and we need to make it very clear >> about when we're pulling the plug on 2.6. > > The moment of switch will be when this unstable becomes a named version, > i.e. stable. However, nothing precludes to still have legacy 2.6 modules > in it. The CSWpy- prefix does preclude 2.6 from having legacy packages, unless we do something specific. Before: CSWpy-foo contains /opt/csw/lib/python/site-packages/foo.py Rebuilding CSWpy-foo with Python 2.7, and: After: CSWpy-foo contains /opt/csw/lib/python2.7/site-packages/foo.py And now the module doesn't work with 2.6 any more. This is the main blocker for using the CSWpy- prefix for 2.7 modules. Back to pyshared. I've read the policy, but it doesn't provide any details. The pyshared functionality might be lurking in their packaging helper tools, dh_python2 and dh_python3. We need to look into how it's done. Until then, we can continue building modules with Python 2.6: they will work with Python 2.6 and 2.7. Maciej From pfelecan at opencsw.org Thu Jul 25 09:55:35 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 25 Jul 2013 09:55:35 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Thu, 25 Jul 2013 08:27:05 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: Thank you for all this work. > And now the module doesn't work with 2.6 any more. This is the main > blocker for using the CSWpy- prefix for 2.7 modules. Then, that is the time for the user to install 2.7. Anyway, when the unstable containing 2.7 transits to a named catalog, i.e. current stable (or what ever its name), it will be the "current" Python. > Back to pyshared. I've read the policy, but it doesn't provide any > details. The pyshared functionality might be lurking in their > packaging helper tools, dh_python2 and dh_python3. We need to look > into how it's done. Until then, we can continue building modules with > Python 2.6: they will work with Python 2.6 and 2.7. "pyshared" is what the French call "intention pieuse" (pious intention), i.e. they have the same reasoning as that that I sustain from the beginning of this thread but inertia and/or other things precluded its usage and what we got is another directory containing versioned sub-directories. I hope that we will not do the same mistake, but the reluctance of using a common prefix for 2.x modules are signs of that. Lets not do the things more complex than needed and use only one prefix for packages containing Python 2.x modules. -- Peter From pfelecan at opencsw.org Thu Jul 25 11:29:12 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 25 Jul 2013 11:29:12 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 22 Jul 2013 11:36:13 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > Just a quick update: I did some testing and it seems like 2.6 with a > versioned directory plus an extra path in the module search path does work. > It's now pushed into unstable ? please test. 1. Installed the new package: PKGINST: CSWpython NAME: python - A high-level scripting language, 2.6 series CATEGORY: application ARCH: i386 VERSION: 2.6.8,REV=2013.07.22 2. alternatives --list ... python-symlink ... Can we use a more "natural" name, such as python? Or, maybe there is a reason for using this name that I didn't get yet... 3. Installed self packaged Python 2.7, revision 21549. Every sub-package containing Python code has some class error: CSWpython27: Compiling py files to normal bytecode ... Error compiling: /opt/csw/lib/python2.7/filecmp.py Error compiling: /opt/csw/lib/python2.7/netrc.py Error compiling: /opt/csw/lib/python2.7/runpy.py Error compiling: /opt/csw/lib/python2.7/subprocess.py Error compiling: /opt/csw/lib/python2.7/zipfile.py Compiling py files to optimized bytecode ... Error compiling: /opt/csw/lib/python2.7/filecmp.py Error compiling: /opt/csw/lib/python2.7/netrc.py Error compiling: /opt/csw/lib/python2.7/runpy.py Error compiling: /opt/csw/lib/python2.7/subprocess.py Error compiling: /opt/csw/lib/python2.7/zipfile.py CSWpython27-tk: Compiling py files to normal bytecode ... Error compiling: /opt/csw/lib/python2.7/lib-tk/test/test_ttk/test_functions.py Compiling py files to optimized bytecode ... Error compiling: /opt/csw/lib/python2.7/lib-tk/test/test_ttk/test_functions.py CSWidle27: Compiling py files to normal bytecode ... Error compiling: /opt/csw/lib/python2.7/idlelib/EditorWindow.py Compiling py files to optimized bytecode ... Error compiling: /opt/csw/lib/python2.7/idlelib/EditorWindow.py Maybe this is due to the "current" Python which is 2.6 according to alternatives --display python-symlink (sic) -- Peter From pfelecan at opencsw.org Thu Jul 25 11:58:36 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 25 Jul 2013 11:58:36 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: (Peter FELECAN's message of "Thu, 25 Jul 2013 11:29:12 +0200") References: Message-ID: Peter FELECAN writes: > "Maciej (Matchek) Blizi?ski" writes: > >> Just a quick update: I did some testing and it seems like 2.6 with a >> versioned directory plus an extra path in the module search path does work. >> It's now pushed into unstable ? please test. > > 1. Installed the new package: > > PKGINST: CSWpython > NAME: python - A high-level scripting language, 2.6 series > CATEGORY: application > ARCH: i386 > VERSION: 2.6.8,REV=2013.07.22 > > 2. alternatives --list > > ... > python-symlink > ... > > Can we use a more "natural" name, such as python? Or, maybe there is > a reason for using this name that I didn't get yet... > > 3. Installed self packaged Python 2.7, revision 21549. > > Every sub-package containing Python code has some class error: > > CSWpython27: > > Compiling py files to normal bytecode ... > Error compiling: /opt/csw/lib/python2.7/filecmp.py > Error compiling: /opt/csw/lib/python2.7/netrc.py > Error compiling: /opt/csw/lib/python2.7/runpy.py > Error compiling: /opt/csw/lib/python2.7/subprocess.py > Error compiling: /opt/csw/lib/python2.7/zipfile.py > Compiling py files to optimized bytecode ... > Error compiling: /opt/csw/lib/python2.7/filecmp.py > Error compiling: /opt/csw/lib/python2.7/netrc.py > Error compiling: /opt/csw/lib/python2.7/runpy.py > Error compiling: /opt/csw/lib/python2.7/subprocess.py > Error compiling: /opt/csw/lib/python2.7/zipfile.py After correcting the alternatives target symbolic link for 2.7, revision 21550, and running alternatives --config python-symlink and choosing 270, the following packages have no more class errors: > CSWpython27-tk: > > Compiling py files to normal bytecode ... > Error compiling: /opt/csw/lib/python2.7/lib-tk/test/test_ttk/test_functions.py > Compiling py files to optimized bytecode ... > Error compiling: /opt/csw/lib/python2.7/lib-tk/test/test_ttk/test_functions.py > > CSWidle27: > > Compiling py files to normal bytecode ... > Error compiling: /opt/csw/lib/python2.7/idlelib/EditorWindow.py > Compiling py files to optimized bytecode ... > Error compiling: /opt/csw/lib/python2.7/idlelib/EditorWindow.py I think that there is something to modify in CSWcswclassutils.i.cswpycompile but, for the moment, I don't have any useful ideas. Maybe some vector to carry the version needed for compiling. As an aside, this brings up the question: why do we need to compile at installation time and not deliver the pre-compiled bits; is the p-code not portable on different architectures? -- Peter From maciej at opencsw.org Thu Jul 25 17:48:48 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Thu, 25 Jul 2013 16:48:48 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/25 Peter FELECAN > > As an aside, this brings up the question: why do we need to > compile at installation time and not deliver the pre-compiled bits; is > the p-code not portable on different architectures? The Debian Python policy doesn't provide the reasons why they do it that way. http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-byte_compilation According to my own web searching, the *.pyc files are Python-version-dependent according to this entry on stackoverflow: http://stackoverflow.com/questions/2263356/are-python-2-5-pyc-files-compatible-with-python-2-6-pyc-files They don't provide sources, so I couldn't quickly verify it. I did see an example where the Python version has been shown to be encoded inside a *.pyc file. I'm not sure how Python handles version mismatches. I'm expecting that it reads the file, deserializes it, check the version and if a mismatch is detected, it discards the *.pyc file and compiles the *.py file. This would mean that a *.pyc file is always useless for one of the two installed Python versions. I looked at the Debian pyshared solution by poking an installed system. The quick summary is: - /usr/share/pyshared contains the .py files and is not a member of sys.path - for each Python version a /usr/lib/pythonX.Y/dist-modules directory exists and is a member of sys.path - when a module is installed, symlinks from /usr/lib/pythonX.Y/dist-modules to respective entries in /usr/share/pyshared are made. They are exclusively file symlinks, and never directory symlinks. - the *.py files are compiled once per Python version, and *.pyc files live in the /usr/lib/pythonX.Y/dist-modules directory tree Sounds very reasonable to me. Somebody has to implement it for us though. Maciej From pfelecan at opencsw.org Thu Jul 25 19:37:01 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Thu, 25 Jul 2013 19:37:01 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Thu, 25 Jul 2013 16:48:48 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/25 Peter FELECAN >> >> As an aside, this brings up the question: why do we need to >> compile at installation time and not deliver the pre-compiled bits; is >> the p-code not portable on different architectures? > > The Debian Python policy doesn't provide the reasons why they do it that way. > http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-byte_compilation > > According to my own web searching, the *.pyc files are > Python-version-dependent according to this entry on stackoverflow: > http://stackoverflow.com/questions/2263356/are-python-2-5-pyc-files-compatible-with-python-2-6-pyc-files > > They don't provide sources, so I couldn't quickly verify it. I did see > an example where the Python version has been shown to be encoded > inside a *.pyc file. I'm not sure how Python handles version > mismatches. I'm expecting that it reads the file, deserializes it, > check the version and if a mismatch is detected, it discards the *.pyc > file and compiles the *.py file. This would mean that a *.pyc file is > always useless for one of the two installed Python versions. Thank you for the research and abstract. All this is not very convincing (beware, I'm not talking about you). I remember a discussion that I had in a distant past with Philip Brown about Emacs Lisp compiled files (.el is the source, .elc is the compiled version); after a lot of wrangling I won by the missing argument for architectural / version specific compiled components and eventually delivered them in the package as it seems suitable from my stand point for Python. > I looked at the Debian pyshared solution by poking an installed > system. The quick summary is: > > - /usr/share/pyshared contains the .py files and is not a member of sys.path > - for each Python version a /usr/lib/pythonX.Y/dist-modules directory > exists and is a member of sys.path > - when a module is installed, symlinks from > /usr/lib/pythonX.Y/dist-modules to respective entries in > /usr/share/pyshared are made. They are exclusively file symlinks, and > never directory symlinks. > - the *.py files are compiled once per Python version, and *.pyc files > live in the /usr/lib/pythonX.Y/dist-modules directory tree This is a genuine example of over-engineering, especially that there is no real argument for doing that (see above). Moreover, if we have separate versioned trees what's the need for that? > Sounds very reasonable to me. Somebody has to implement it for us though. IMHO we don't have the resources for that. However, someone willing to spend his time on it is free to do it. In the mean time, what we must do is to find a solution for the compilation errors during the installation of some Python modules, as reported in the original message. -- Peter From maciej at opencsw.org Thu Jul 25 20:24:00 2013 From: maciej at opencsw.org (Maciej =?utf-8?Q?Blizi=C5=84ski?=) Date: Thu, 25 Jul 2013 19:24:00 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: <20130725182359.GA20759@cotton.home.blizinski.pl> On Thu, Jul 25, 2013 at 07:37:01PM +0200, Peter FELECAN wrote: > "Maciej (Matchek) Blizi?ski" writes: > > > > They don't provide sources, so I couldn't quickly verify it. I did see > > an example where the Python version has been shown to be encoded > > inside a *.pyc file. I'm not sure how Python handles version > > mismatches. I'm expecting that it reads the file, deserializes it, > > check the version and if a mismatch is detected, it discards the *.pyc > > file and compiles the *.py file. This would mean that a *.pyc file is > > always useless for one of the two installed Python versions. > > Thank you for the research and abstract. All this is not very convincing > (beware, I'm not talking about you). I remember a discussion that I had > in a distant past with Philip Brown about Emacs Lisp compiled files (.el is the source, > .elc is the compiled version); after a lot of wrangling I won by the > missing argument for architectural / version specific compiled > components and eventually delivered them in the package as it seems > suitable from my stand point for Python. I found a quote from Python docs: http://docs.python.org/2/glossary.html#term-bytecode """Do note that bytecodes are not expected to work between different Python virtual machines, nor to be stable between Python releases.""" Just to add weight to the argument that .pyc files are not portable across Python releases (e.g. 2.6 and 2.7). What are you thinking of doing then, delivering bytecode for one specific python version and rendering the *.pyc files useless for other versions? > > I looked at the Debian pyshared solution by poking an installed > > system. The quick summary is: > > > > - /usr/share/pyshared contains the .py files and is not a member of sys.path > > - for each Python version a /usr/lib/pythonX.Y/dist-modules directory > > exists and is a member of sys.path > > - when a module is installed, symlinks from > > /usr/lib/pythonX.Y/dist-modules to respective entries in > > /usr/share/pyshared are made. They are exclusively file symlinks, and > > never directory symlinks. > > - the *.py files are compiled once per Python version, and *.pyc files > > live in the /usr/lib/pythonX.Y/dist-modules directory tree > > This is a genuine example of over-engineering, especially that there is > no real argument for doing that (see above). If you want the modules to be shared across 2.6 and 2.7, and you already know that you cannot share the *.pyc files across 2.6 and 2.7, why do you see no real argument? If we ship useless *.pyc files, we can just skip shipping them. > Moreover, if we have separate versioned trees what's the need for > that? You want to share modules between 2.6 and 2.7, right? Then you need to keep the *.py files in a shared place, and keep *.pyc files in a non-shared place. > > Sounds very reasonable to me. Somebody has to implement it for us though. > > IMHO we don't have the resources for that. However, someone willing to > spend his time on it is free to do it. > > In the mean time, what we must do is to find a solution for the > compilation errors during the installation of some Python modules, as > reported in the original message. The pycompile classaction must be aware of the Python version for which the module is built and invoke the right interpreter. I have to say these errors seem to disprove the "2.6 and 2.7 are totally compatible" claim. Or is there a different (than the file contents) reason for these failures, do you know? Maciej From pfelecan at opencsw.org Fri Jul 26 09:36:39 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 26 Jul 2013 09:36:39 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: <20130725182359.GA20759@cotton.home.blizinski.pl> ("Maciej =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Thu, 25 Jul 2013 19:24:00 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: Maciej Blizi?ski writes: > On Thu, Jul 25, 2013 at 07:37:01PM +0200, Peter FELECAN wrote: >> "Maciej (Matchek) Blizi?ski" writes: >> > >> > They don't provide sources, so I couldn't quickly verify it. I did see >> > an example where the Python version has been shown to be encoded >> > inside a *.pyc file. I'm not sure how Python handles version >> > mismatches. I'm expecting that it reads the file, deserializes it, >> > check the version and if a mismatch is detected, it discards the *.pyc >> > file and compiles the *.py file. This would mean that a *.pyc file is >> > always useless for one of the two installed Python versions. >> >> Thank you for the research and abstract. All this is not very convincing >> (beware, I'm not talking about you). I remember a discussion that I had >> in a distant past with Philip Brown about Emacs Lisp compiled files (.el is the source, >> .elc is the compiled version); after a lot of wrangling I won by the >> missing argument for architectural / version specific compiled >> components and eventually delivered them in the package as it seems >> suitable from my stand point for Python. > > I found a quote from Python docs: > http://docs.python.org/2/glossary.html#term-bytecode > > """Do note that bytecodes are not expected to work between different Python > virtual machines, nor to be stable between Python releases.""" > > Just to add weight to the argument that .pyc files are not portable > across Python releases (e.g. 2.6 and 2.7). > > What are you thinking of doing then, delivering bytecode for one > specific python version and rendering the *.pyc files useless for other > versions? >> > I looked at the Debian pyshared solution by poking an installed >> > system. The quick summary is: >> > >> > - /usr/share/pyshared contains the .py files and is not a member of sys.path >> > - for each Python version a /usr/lib/pythonX.Y/dist-modules directory >> > exists and is a member of sys.path >> > - when a module is installed, symlinks from >> > /usr/lib/pythonX.Y/dist-modules to respective entries in >> > /usr/share/pyshared are made. They are exclusively file symlinks, and >> > never directory symlinks. >> > - the *.py files are compiled once per Python version, and *.pyc files >> > live in the /usr/lib/pythonX.Y/dist-modules directory tree >> >> This is a genuine example of over-engineering, especially that there is >> no real argument for doing that (see above). > > If you want the modules to be shared across 2.6 and 2.7, and you already > know that you cannot share the *.pyc files across 2.6 and 2.7, why do > you see no real argument? > If we ship useless *.pyc files, we can just skip shipping them. Right. >> Moreover, if we have separate versioned trees what's the need for >> that? > > You want to share modules between 2.6 and 2.7, right? Then you need to > keep the *.py files in a shared place, and keep *.pyc files in > a non-shared place. Right. >> > Sounds very reasonable to me. Somebody has to implement it for us though. >> >> IMHO we don't have the resources for that. However, someone willing to >> spend his time on it is free to do it. >> >> In the mean time, what we must do is to find a solution for the >> compilation errors during the installation of some Python modules, as >> reported in the original message. > > The pycompile classaction must be aware of the Python version for which > the module is built and invoke the right interpreter. > > I have to say these errors seem to disprove the "2.6 and 2.7 are totally > compatible" claim. Or is there a different (than the file contents) > reason for these failures, do you know? I thought that you'll explore the issue... -- Peter From maciej at opencsw.org Fri Jul 26 10:40:41 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 26 Jul 2013 09:40:41 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/26 Peter FELECAN > I thought that you'll explore the issue... I looked at our pycompile class action script. The first quick modification was to make it stop hiding causes of compilation failures. The next modification will be to never invoke /opt/csw/bin/python, but a specific version of the interpreter instead. The tricky part of that is knowing which interpreter to invoke. I'm thinking of simple pattern matching: /opt/csw/lib/python/ ? 2.6 /opt/csw/lib/pythonX.Y/ ? X.Y Maciej From pfelecan at opencsw.org Fri Jul 26 11:13:09 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 26 Jul 2013 11:13:09 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Fri, 26 Jul 2013 09:40:41 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/26 Peter FELECAN >> I thought that you'll explore the issue... > > I looked at our pycompile class action script. The first quick > modification was to make it stop hiding causes of compilation > failures. > > The next modification will be to never invoke /opt/csw/bin/python, but > a specific version of the interpreter instead. The tricky part of that > is knowing which interpreter to invoke. I'm thinking of simple pattern > matching: > > /opt/csw/lib/python/ ? 2.6 > /opt/csw/lib/pythonX.Y/ ? X.Y Ah, Maciej! You still wish to freeze the time to 2.6... Joke apart, if we think longer term and generic, isn't there a way to convey the version in the package? Another possibility: use specific class action scripts for each version; I'm quite cold with this one but mindenfeltev?stkimer?t? tend to be my unfaithful mistress. -- Peter From pfelecan at opencsw.org Fri Jul 26 12:12:17 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 26 Jul 2013 12:12:17 +0200 Subject: [csw-maintainers] reinplace before extract? Message-ID: I observe a very strange phenomenon: in the qt4-gcc recipe there is the following: REINPLACE_USRLOCAL += src/gui/kernel/qguiplatformplugin.cpp REINPLACE_USRLOCAL += src/network/ssl/qsslsocket_openssl_symbols.cpp REINPLACE_USRLOCAL += src/network/ssl/qsslsocket_openssl.cpp # replace SSL cert location REINPLACE_USRLOCAL += src/network/ssl/qsslsocket.cpp REINPLACE_USRLOCAL += src/3rdparty/webkit/Source/WebCore/plugins/PluginDatabase.cpp when doing a mgar clean package, I get: ==> Copying work/download/qt-everywhere-opensource-src-4.8.5.tar.gz perl -p -i -e 's{/usr/local}{/opt/csw}g' \ work/build-global/qt-everywhere-opensource-src-4.8.5/src/gui/kernel/qguiplatformplugin.cpp work/build-global/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket_openssl_symbols.cpp work/build-global/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket_openssl.cpp work/build-global/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket.cpp work/build-global/qt-everywhere-opensource-src-4.8.5/src/3rdparty/webkit/Source/WebCore/plugins/PluginDatabase.cpp Can't open work/build-global/qt-everywhere-opensource-src-4.8.5/src/gui/kernel/qguiplatformplugin.cpp: No such file or directory. Can't open work/build-global/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket_openssl_symbols.cpp: No such file or directory. Can't open work/build-global/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket_openssl.cpp: No such file or directory. Can't open work/build-global/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket.cpp: No such file or directory. Can't open work/build-global/qt-everywhere-opensource-src-4.8.5/src/3rdparty/webkit/Source/WebCore/plugins/PluginDatabase.cpp: No such file or directory. [extract-modulated] complete for qt. and: ==> Extracting work/download/qt-everywhere-opensource-src-4.8.5.tar.gz perl -p -i -e 's{/usr/local}{/opt/csw}g' \ work/build-isa-pentium_pro/qt-everywhere-opensource-src-4.8.5/src/gui/kernel/qguiplatformplugin.cpp work/build-isa-pentium_pro/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket_openssl_symbols.cpp work/build-isa-pentium_pro/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket_openssl.cpp work/build-isa-pentium_pro/qt-everywhere-opensource-src-4.8.5/src/network/ssl/qsslsocket.cpp work/build-isa-pentium_pro/qt-everywhere-opensource-src-4.8.5/src/3rdparty/webkit/Source/WebCore/plugins/PluginDatabase.cpp [extract-modulated] complete for qt. What intrigues me is that I have 2 instances of the reinplace call: one before the extraction and the other one after; however, there are to instances of the information message: "[extract-modulated] complete for qt." In the end the reimplacement is executed but the initial error messages are disturbing. Can a knowledgeable soul explore and explain? TIA -- Peter From maciej at opencsw.org Fri Jul 26 12:54:30 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 26 Jul 2013 11:54:30 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/26 Peter FELECAN : > Ah, Maciej! You still wish to freeze the time to 2.6... /opt/csw/lib/python is not a generic location, but a specifically 2.6 location, right? > Joke apart, if > we think longer term and generic, isn't there a way to convey the > version in the package? In general, one package could provide files for more than one version of Python, so the granularity has to be at the individual file level. > Another possibility: use specific class action > scripts for each version; I'm quite cold with this one but > mindenfeltev?stkimer?t? tend to be my unfaithful mistress. This would also be an option. You could define the regex patterns in GAR. But I'm thinking about sewing that logic into the CAS as the first option. Maciej From pfelecan at opencsw.org Fri Jul 26 13:17:40 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 26 Jul 2013 13:17:40 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Fri, 26 Jul 2013 11:54:30 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > But I'm thinking about sewing that logic into the CAS as the first > option. I'm curious to see that. -- Peter From maciej at opencsw.org Fri Jul 26 15:02:49 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Fri, 26 Jul 2013 14:02:49 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/26 Peter FELECAN > > > I have to say these errors seem to disprove the "2.6 and 2.7 are totally > > compatible" claim. Or is there a different (than the file contents) > > reason for these failures, do you know? > > I thought that you'll explore the issue... The newly updated CAS displays underlying errors. To answer my own question, these errors occur when compiling 2.7 core modules with the 2.6 interpreter. The failures reported are all about the new bits of syntax. The new CAS package is now in unstable. Maciej From dam at opencsw.org Fri Jul 26 17:00:27 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Fri, 26 Jul 2013 17:00:27 +0200 Subject: [csw-maintainers] reinplace before extract? In-Reply-To: References: Message-ID: Hi Peter, Am 26.07.2013 um 12:12 schrieb Peter FELECAN : > I observe a very strange phenomenon: in the qt4-gcc recipe there is the > following: ? > What intrigues me is that I have 2 instances of the reinplace call: one > before the extraction and the other one after; however, there are to > instances of the information message: "[extract-modulated] complete for > qt." > > In the end the reimplacement is executed but the initial error messages > are disturbing. Short answer: This is safe to ignore. > Can a knowledgeable soul explore and explain? Sure. This is due to modulations, some phases are done in global modulation, some in isa-specific modulations. In the global modulation all files from DISTFILES are *copied* to $(WORKDIR), in isa-specific modulations these files are *unpacked* to the isa-specific $(WORKDIR). As reinplacements are invoked after the extract phase they may fail if they are applied to files resulting from unpacking in global context. The rationale is that it may be necessary to put files from DISTFILES verbatim in a package, this is doable by copying them during merge from global phase to $(PKGROOT). An alternative solution would be to specifically define to which phase you would like to apply the reinplacement. Please see also http://sourceforge.net/apps/trac/gar/wiki/Modulations Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From pfelecan at opencsw.org Fri Jul 26 18:54:12 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 26 Jul 2013 18:54:12 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Fri, 26 Jul 2013 14:02:49 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > The newly updated CAS displays underlying errors. To answer my own > question, these errors occur when compiling 2.7 core modules with the > 2.6 interpreter. The failures reported are all about the new bits of > syntax. Which is nominal. The reverse would be not. But, solving the propagation of the interpreter version to the CAS script will solve out issue, isn't it? -- Peter From pfelecan at opencsw.org Fri Jul 26 18:58:24 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Fri, 26 Jul 2013 18:58:24 +0200 Subject: [csw-maintainers] reinplace before extract? In-Reply-To: (Dagobert Michelsen's message of "Fri, 26 Jul 2013 17:00:27 +0200") References: Message-ID: Dagobert Michelsen writes: > Hi Peter, > > Am 26.07.2013 um 12:12 schrieb Peter FELECAN : > >> I observe a very strange phenomenon: in the qt4-gcc recipe there is the >> following: > ? >> What intrigues me is that I have 2 instances of the reinplace call: one >> before the extraction and the other one after; however, there are to >> instances of the information message: "[extract-modulated] complete for >> qt." >> >> In the end the reimplacement is executed but the initial error messages >> are disturbing. > > Short answer: This is safe to ignore. > >> Can a knowledgeable soul explore and explain? > > > Sure. This is due to modulations, some phases are done in global modulation, > some in isa-specific modulations. In the global modulation all files from DISTFILES > are *copied* to $(WORKDIR), in isa-specific modulations these files are *unpacked* > to the isa-specific $(WORKDIR). As reinplacements are invoked after the extract > phase they may fail if they are applied to files resulting from unpacking in > global context. The rationale is that it may be necessary to put files from DISTFILES > verbatim in a package, this is doable by copying them during merge from global > phase to $(PKGROOT). Thank you Dago for the explanation. Frankly, I don't like this kind of pseudo-errors. Is there a simple way to avoid them? For example, to enrich the definition domain for the WHEN clause of reinplace. > An alternative solution would be to specifically define to which phase you would > like to apply the reinplacement. That is. As a mgar user, can I do something in the recipe for that, or a divine intervention is necessary? See above suggestion. -- Peter From dam at opencsw.org Fri Jul 26 19:53:13 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Fri, 26 Jul 2013 19:53:13 +0200 Subject: [csw-maintainers] reinplace before extract? In-Reply-To: References: Message-ID: <70FE700B-6F3D-4C98-A1DB-19F8D408F57D@opencsw.org> Hi Peter, Am 26.07.2013 um 18:58 schrieb Peter FELECAN : > Dagobert Michelsen writes: >> Am 26.07.2013 um 12:12 schrieb Peter FELECAN : >> >>> I observe a very strange phenomenon: in the qt4-gcc recipe there is the >>> following: >> ? >>> What intrigues me is that I have 2 instances of the reinplace call: one >>> before the extraction and the other one after; however, there are to >>> instances of the information message: "[extract-modulated] complete for >>> qt." >>> >>> In the end the reimplacement is executed but the initial error messages >>> are disturbing. >> >> Short answer: This is safe to ignore. >> >>> Can a knowledgeable soul explore and explain? >> >> Sure. This is due to modulations, some phases are done in global modulation, >> some in isa-specific modulations. In the global modulation all files from DISTFILES >> are *copied* to $(WORKDIR), in isa-specific modulations these files are *unpacked* >> to the isa-specific $(WORKDIR). As reinplacements are invoked after the extract >> phase they may fail if they are applied to files resulting from unpacking in >> global context. The rationale is that it may be necessary to put files from DISTFILES >> verbatim in a package, this is doable by copying them during merge from global >> phase to $(PKGROOT). > > Thank you Dago for the explanation. Frankly, I don't like this kind of > pseudo-errors. Is there a simple way to avoid them? For example, to > enrich the definition domain for the WHEN clause of reinplace. I would think of REINPLACE_MODULATION_ += global REINPLACE_MODULATION_ += default REINPLACE_MODULATION_ += default64 REINPLACE_MODULATION_ += isa-sparcv8plus >> An alternative solution would be to specifically define to which phase you would >> like to apply the reinplacement. > > That is. As a mgar user, can I do something in the recipe for that, or a > divine intervention is necessary? See above suggestion. The addition isn't too hard, I'll see that I do that next week. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From pfelecan at opencsw.org Sat Jul 27 10:42:11 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Sat, 27 Jul 2013 10:42:11 +0200 Subject: [csw-maintainers] reinplace before extract? In-Reply-To: <70FE700B-6F3D-4C98-A1DB-19F8D408F57D@opencsw.org> (Dagobert Michelsen's message of "Fri, 26 Jul 2013 19:53:13 +0200") References: <70FE700B-6F3D-4C98-A1DB-19F8D408F57D@opencsw.org> Message-ID: Dagobert Michelsen writes: >> Thank you Dago for the explanation. Frankly, I don't like this kind of >> pseudo-errors. Is there a simple way to avoid them? For example, to >> enrich the definition domain for the WHEN clause of reinplace. > > I would think of > > REINPLACE_MODULATION_ += global > REINPLACE_MODULATION_ += default > REINPLACE_MODULATION_ += default64 > REINPLACE_MODULATION_ += isa-sparcv8plus > >>> An alternative solution would be to specifically define to which phase you would >>> like to apply the reinplacement. >> >> That is. As a mgar user, can I do something in the recipe for that, or a >> divine intervention is necessary? See above suggestion. > > The addition isn't too hard, I'll see that I do that next week. Good. Don't forget to document it. When it's done, I would like to test it. Thank you again. -- Peter From rupert at opencsw.org Sun Jul 28 12:14:55 2013 From: rupert at opencsw.org (rupert THURNER) Date: Sun, 28 Jul 2013 12:14:55 +0200 Subject: [csw-maintainers] upgrade sourceforge? Message-ID: hi, committing to sourceforge gives an upgrade warning. do you plan to upgrade? rupert. From maciej at opencsw.org Sun Jul 28 12:18:55 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sun, 28 Jul 2013 11:18:55 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/26 Peter FELECAN > > > The newly updated CAS displays underlying errors. To answer my own > > question, these errors occur when compiling 2.7 core modules with the > > 2.6 interpreter. The failures reported are all about the new bits of > > syntax. > > Which is nominal. The reverse would be not. But, solving the propagation > of the interpreter version to the CAS script will solve out issue, isn't > it? Yes, but it would be interpreter versions, in plural, because one package could contain files for more than one version of the interpreter. I rewrote the pycompile CAS to support multiple Python versions, using very simple pattern matching. http://sourceforge.net/apps/trac/gar/changeset/21563 (I wrote 2012 by mistake. Already corrected.) It passes the smoke test on my VM, I'm pushing it to unstable. I wanted to avoid using files for caching file lists, but I think I'll revert it back to a state where lists of files to compile are saved. The reason is that it current starts calling the interpreter before all files are saved, so some of the compilation invocations are failing. This should be an easy fix. Maciej From pfelecan at opencsw.org Sun Jul 28 13:55:13 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Sun, 28 Jul 2013 13:55:13 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Sun, 28 Jul 2013 11:18:55 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/26 Peter FELECAN >> >> > The newly updated CAS displays underlying errors. To answer my own >> > question, these errors occur when compiling 2.7 core modules with the >> > 2.6 interpreter. The failures reported are all about the new bits of >> > syntax. >> >> Which is nominal. The reverse would be not. But, solving the propagation >> of the interpreter version to the CAS script will solve out issue, isn't >> it? > > Yes, but it would be interpreter versions, in plural, because one > package could contain files for more than one version of the > interpreter. > > I rewrote the pycompile CAS to support multiple Python versions, using > very simple pattern matching. > http://sourceforge.net/apps/trac/gar/changeset/21563 (I wrote 2012 by > mistake. Already corrected.) > > It passes the smoke test on my VM, I'm pushing it to unstable. > > I wanted to avoid using files for caching file lists, but I think I'll > revert it back to a state where lists of files to compile are saved. > The reason is that it current starts calling the interpreter before > all files are saved, so some of the compilation invocations are > failing. This should be an easy fix. I already read the code and it's a nice piece of engineering. It's not exactly what I had in mind or wrote myself. We will test it throughly when the first complete implementation is finished. -- Peter From rupert at opencsw.org Sun Jul 28 18:51:40 2013 From: rupert at opencsw.org (rupert THURNER) Date: Sun, 28 Jul 2013 18:51:40 +0200 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: References: Message-ID: hi, i try to build a new serf version ... and i am wondering why it says /usr/bin/env: No such file or directory or am i misreading this? rupert @ unstable10x : ~/opencsw/libserf/trunk ... cd work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0 && mkdir -p . && cd . && /usr/bin/env -i HOME="/home/rupert" PATH="/home/rupert/opencsw/.buildsys/v2/gar/bin/sos12-wrappers:/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/install-isa-pentium_pro/opt/csw/bin:/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/install-isa-pentium_pro/opt/csw/bin:/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/install-isa-pentium_pro/opt/csw/sbin:/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/install-isa-pentium_pro/opt/csw/sbin:/opt/csw/bin:/opt/csw/bin:/opt/csw/sbin:/opt/csw/sbin:/opt/SUNWspro/bin:/home/rupert/opencsw/.buildsys/v2/gar/bin:/usr/bin:/usr/sbin:/usr/java/bin:/usr/ccs/bin:/usr/openwin/bin" LC_ALL="C" prefix="/opt/csw" exec_prefix="/opt/csw" bindir="/opt/csw/bin" sbindir="/opt/csw/sbin" libexecdir="/opt/csw/libexec" datadir="/opt/csw/share" sysconfdir="/etc/opt/csw" sharedstatedir="/opt/csw/share" localstatedir="/var/opt/csw" libdir="/opt/csw/lib" infodir="/opt/csw/share/info" lispdir="/opt/csw/share/emacs/site-lisp" includedir="/opt/csw/include" mandir="/opt/csw/share/man" docdir="/opt/csw/share/doc" sourcedir="/opt/csw/src" CPPFLAGS="-I/opt/csw/include" CFLAGS="-xO3 -m32 -xarch=pentium_pro -xchip=pentium_pro" CXXFLAGS="-xO3 -m32 -xarch=pentium_pro -xchip=pentium_pro" LDFLAGS="-m32 -xarch=pentium_pro -xchip=pentium_pro -L/opt/csw/bdb48/lib -L/opt/csw/lib" FFLAGS="-xO3 -m32 -xarch=pentium_pro -xchip=pentium_pro" FCFLAGS="-xO3 -m32 -xarch=pentium_pro -xchip=pentium_pro" F77="/opt/SUNWspro/bin/f77" FC="/opt/SUNWspro/bin/f95" ASFLAGS="" OPTFLAGS="-xO3 -m32 -xarch=pentium_pro -xchip=pentium_pro" CC="/opt/SUNWspro/bin/cc" CXX="/opt/SUNWspro/bin/CC" CC_HOME="/opt/SUNWspro" CC_VERSION="Sun C 5.9 SunOS_i386 Patch 124868-15 2010/08/11" CXX_VERSION="Sun C++ 5.9 SunOS_i386 Patch 124864-30 2012/07/11" GARCH="i386" GAROSREL="5.10" GARPACKAGE="trunk" LD_OPTIONS="-R/opt/csw/bdb48/lib/\$ISALIST -R/opt/csw/bdb48/lib -R/opt/csw/lib/\$ISALIST -R/opt/csw/lib -M /home/rupert/opencsw/.buildsys/v2/gar/lib/map.solaris10 -B direct -z ignore" PKG_CONFIG_PATH="/opt/csw/lib/pkgconfig" DESTDIR="/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/install-isa-pentium_pro" /home/rupert/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/configure --prefix=/opt/csw --exec_prefix=/opt/csw --bindir=/opt/csw/bin --sbindir=/opt/csw/sbin --libexecdir=/opt/csw/libexec --datadir=/opt/csw/share --sysconfdir=/etc/opt/csw --sharedstatedir=/opt/csw/share --localstatedir=/var/opt/csw --libdir=/opt/csw/lib --infodir=/opt/csw/share/info --includedir=/opt/csw/include --mandir=/opt/csw/share/man --with-apr=/opt/csw/bin/apr-1-config --with-apr-util=/opt/csw/bin/apu-1-config /usr/bin/env: No such file or directory gmake[1]: *** [configure-work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/configure] Error 127 gmake[1]: Leaving directory `/home/rupert/opencsw/libserf/trunk' From maciej at opencsw.org Sun Jul 28 18:53:45 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sun, 28 Jul 2013 17:53:45 +0100 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: References: Message-ID: It's /usr/bin/env telling you that the executable you wanted it to run, does not exist in this case, it's /home/rupert/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/configure -- right in the middle between the environment variables and command line options. From rupert at opencsw.org Sun Jul 28 20:01:44 2013 From: rupert at opencsw.org (rupert THURNER) Date: Sun, 28 Jul 2013 20:01:44 +0200 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: References: Message-ID: yes, i see they switched to scons. should mgar handle that automatically, or what do you expect me to do in this case? when i run rupert @ unstable10x : ~/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0 $ scons -Q scons: *** Directory path for option PREFIX does not exist: /usr/local File "/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/SConstruct", line 133, in On Sun, Jul 28, 2013 at 6:53 PM, Maciej (Matchek) Blizi?ski wrote: > It's /usr/bin/env telling you that the executable you wanted it to > run, does not exist in this case, it's > /home/rupert/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/configure > -- right in the middle between the environment variables and command > line options. > _______________________________________________ > maintainers mailing list > maintainers at lists.opencsw.org > https://lists.opencsw.org/mailman/listinfo/maintainers > .:: This mailing list's archive is public. ::. From maciej at opencsw.org Sun Jul 28 23:33:36 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sun, 28 Jul 2013 22:33:36 +0100 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: References: Message-ID: 2013/7/28 rupert THURNER : > yes, i see they switched to scons. should mgar handle that > automatically, or what do you expect me to do in this case? when i run > rupert @ unstable10x : > ~/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0 > $ scons -Q > > scons: *** Directory path for option PREFIX does not exist: /usr/local > File "/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/SConstruct", > line 133, in Generally, when you take on builds with anything that isn't autotools, things never work out of the box. You need to work out a way to compile the sources and then encode them in the Makefile. If you're looking for possible things to do, you can grep existing Makefiles for scons, maybe something interesting will show up? Maciej From maciej at opencsw.org Mon Jul 29 00:25:53 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Sun, 28 Jul 2013 23:25:53 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/28 Peter FELECAN : > I already read the code and it's a nice piece of engineering. It's not > exactly what I had in mind or wrote myself. We will test it throughly > when the first complete implementation is finished. All the basic pieces are now in place. I dropped the fifos in favor of regular files. I kept the file descriptors to avoid repeatedly opening and closing the temporary files. I'm happy with the way it is now, we can proceed with testing. I did not implement compiling one .py file multiple times for different Python versions. The main concern for the current CAS is the suitability of pattern matching. Is it good enough to just match a part of the path to know which interpreter to use? It should be good enough for most packages, but I'm not sure what the corner cases / contrived path names could be. Maciej From rupert at opencsw.org Mon Jul 29 01:01:06 2013 From: rupert at opencsw.org (rupert THURNER) Date: Mon, 29 Jul 2013 01:01:06 +0200 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: References: Message-ID: On Sun, Jul 28, 2013 at 11:33 PM, Maciej (Matchek) Blizi?ski wrote: > 2013/7/28 rupert THURNER : >> yes, i see they switched to scons. should mgar handle that >> automatically, or what do you expect me to do in this case? when i run >> rupert @ unstable10x : >> ~/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0 >> $ scons -Q >> >> scons: *** Directory path for option PREFIX does not exist: /usr/local >> File "/home/rupert/opencsw/libserf/trunk/work/solaris10-i386/build-isa-pentium_pro/serf-1.3.0/SConstruct", >> line 133, in > > Generally, when you take on builds with anything that isn't autotools, > things never work out of the box. You need to work out a way to > compile the sources and then encode them in the Makefile. If you're > looking for possible things to do, you can grep existing Makefiles for > scons, maybe something interesting will show up? i found mongodb, and i tried to convert the Makefile to call scons with the correct options. currently i am stuck at: scons: Building targets ... cc -o context.o -c -std=c89 -Wdeclaration-after-statement -Wmissing-prototypes -O2 -mt -DNDEBUG -DSOLARIS2=9 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I. -I/opt/csw/include -I/opt/csw/bdb48/include -I/usr/include context.c cc: Warning: illegal option -d=c89 cc: illegal option -Wdeclaration-after-statement scons: *** [context.o] Error 1 SConstruct contains this in the lines of: if sys.platform != 'win32': env.Append(CFLAGS='-std=c89') env.Append(CCFLAGS=[ '-Wdeclaration-after-statement', '-Wmissing-prototypes', ]) rupert From maciej at opencsw.org Mon Jul 29 09:13:39 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 08:13:39 +0100 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: References: Message-ID: 2013/7/29 rupert THURNER : > i found mongodb, and i tried to convert the Makefile to call scons > with the correct options. currently i am stuck at: > > scons: Building targets ... > cc -o context.o -c -std=c89 -Wdeclaration-after-statement > -Wmissing-prototypes -O2 -mt -DNDEBUG -DSOLARIS2=9 > -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I. > -I/opt/csw/include -I/opt/csw/bdb48/include -I/usr/include context.c > cc: Warning: illegal option -d=c89 > cc: illegal option -Wdeclaration-after-statement > scons: *** [context.o] Error 1 > > SConstruct contains this in the lines of: > if sys.platform != 'win32': > env.Append(CFLAGS='-std=c89') > env.Append(CCFLAGS=[ > '-Wdeclaration-after-statement', > '-Wmissing-prototypes', > ]) Looks like the build system assumes without testing that the compiler supports the listed flags. You have three options: 1. Patch the libserf build system so that it does not unconditionally add these flags to compiler invocations 2. Change the compiler to one that supports these flags 3. Talk to upstream about modifying the libserf build system to support our environment The third option usually requires the most effort, but is the best one in the long run. Maciej From pfelecan at opencsw.org Mon Jul 29 09:37:32 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 09:37:32 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Sun, 28 Jul 2013 23:25:53 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/28 Peter FELECAN : >> I already read the code and it's a nice piece of engineering. It's not >> exactly what I had in mind or wrote myself. We will test it throughly >> when the first complete implementation is finished. > > All the basic pieces are now in place. I dropped the fifos in favor of > regular files. I kept the file descriptors to avoid repeatedly opening > and closing the temporary files. I'm happy with the way it is now, we > can proceed with testing. > > I did not implement compiling one .py file multiple times for > different Python versions. Do you plan to do it later? > The main concern for the current CAS is the suitability of pattern > matching. Is it good enough to just match a part of the path to know > which interpreter to use? It should be good enough for most packages, > but I'm not sure what the corner cases / contrived path names could > be. Even if it misses some is not blocking as the code is there. Anyhow, the difference between .py and .pyc is not so great (and I don't speak about .pyo which seems to me a scam), in my experience and if I understood the mechanism of dynamic compilation the overhead is incurred only once. This is why supplying the compiled components in the package seemed to me a very good solution. -- Peter From pfelecan at opencsw.org Mon Jul 29 09:41:23 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 09:41:23 +0200 Subject: [csw-maintainers] Fwd: /usr/bin/env: No such file or directory In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 08:13:39 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 rupert THURNER : >> i found mongodb, and i tried to convert the Makefile to call scons >> with the correct options. currently i am stuck at: >> >> scons: Building targets ... >> cc -o context.o -c -std=c89 -Wdeclaration-after-statement >> -Wmissing-prototypes -O2 -mt -DNDEBUG -DSOLARIS2=9 >> -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I. >> -I/opt/csw/include -I/opt/csw/bdb48/include -I/usr/include context.c >> cc: Warning: illegal option -d=c89 >> cc: illegal option -Wdeclaration-after-statement >> scons: *** [context.o] Error 1 >> >> SConstruct contains this in the lines of: >> if sys.platform != 'win32': >> env.Append(CFLAGS='-std=c89') >> env.Append(CCFLAGS=[ >> '-Wdeclaration-after-statement', >> '-Wmissing-prototypes', >> ]) > > Looks like the build system assumes without testing that the compiler > supports the listed flags. You have three options: > > 1. Patch the libserf build system so that it does not unconditionally add > these flags to compiler invocations > 2. Change the compiler to one that supports these flags > 3. Talk to upstream about modifying the libserf build system to > support our environment > > The third option usually requires the most effort, but is the best one > in the long run. And the second one is the more adequate in our context. I still don't get why people bother to use SUN Studio. Because there is a choice? -- Peter From rupert at opencsw.org Mon Jul 29 10:37:43 2013 From: rupert at opencsw.org (rupert THURNER) Date: Mon, 29 Jul 2013 10:37:43 +0200 Subject: [csw-maintainers] Fwd: [serf-dev] building on solaris In-Reply-To: References: <27657294-cd9a-46b4-818f-3d0ba6a88e0e@googlegroups.com> Message-ID: hi, fyi, gregs response. somebody of you could help here as well? rupert. ---------- Forwarded message ---------- From: Greg Stein Date: Mon, Jul 29, 2013 at 10:26 AM Subject: Re: [serf-dev] building on solaris To: serf-dev at googlegroups.com, rupert.thurner at gmail.com On Mon, Jul 29, 2013 at 4:03 AM, rupert.thurner at gmail.com wrote: > hi, i m helping to maintain serf for the opencsw solaris build: > http://www.opencsw.org/packages/CSWlibserf1-0/ Geez, I hate APR. Looking at that package dependency list... there is no reason that trying to use serf should require LDAP and BDB. Gah!!! > now than the build changed to scons, i have some difficulties to get it to > compile again with an error: > >>> scons: Building targets ... >>> cc -o context.o -c -std=c89 -Wdeclaration-after-statement >>> -Wmissing-prototypes -O2 -mt -DNDEBUG -DSOLARIS2=9 >>> -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_LARGEFILE64_SOURCE -I. >>> -I/opt/csw/include -I/opt/csw/bdb48/include -I/usr/include context.c >>> cc: Warning: illegal option -d=c89 >>> cc: illegal option -Wdeclaration-after-statement >>> scons: *** [context.o] Error 1 >>> >>> SConstruct contains this in the lines of: >>> if sys.platform != 'win32': >>> env.Append(CFLAGS='-std=c89') >>> env.Append(CCFLAGS=[ >>> '-Wdeclaration-after-statement', >>> '-Wmissing-prototypes', >>> ]) > > we use a Makefile based build system called mgar, for serf it looks like > this: > http://sourceforge.net/apps/trac/gar/browser/csw/mgar/pkg/libserf/trunk/Makefile > > as scons dummy user i am a little unsure how to either convince it to use > gcc > $ which gcc > /opt/csw/bin/gcc > > or to convince it to use the compile flags of the solaris default compiler. These kinds of refinements are just what we're looking for. I had access to a sunos5 box, and it (theoretically) built there. But that was a long while back. I'm trying to get back on that box, and will refine the SConstruct to build there. My hope would be to build on the native cc. If somebody tells it to use gcc... that will likely be a bit more work. We'll be releasing a 1.3.1 soon, and any suggestions/patches you may have will be helpful. I'll try to get the build working on my solaris box, and hope that it works for you too. Thanks, -g From maciej at opencsw.org Mon Jul 29 10:37:28 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 09:37:28 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN : > "Maciej (Matchek) Blizi?ski" writes: >> I did not implement compiling one .py file multiple times for >> different Python versions. > > Do you plan to do it later? No, unless we figure out and agree on how to do it. >> The main concern for the current CAS is the suitability of pattern >> matching. Is it good enough to just match a part of the path to know >> which interpreter to use? It should be good enough for most packages, >> but I'm not sure what the corner cases / contrived path names could >> be. > > Even if it misses some is not blocking as the code is there. Anyhow, the > difference between .py and .pyc is not so great (and I don't speak about > .pyo which seems to me a scam), in my experience and if I understood the > mechanism of dynamic compilation the overhead is incurred only > once. This is why supplying the compiled components in the package > seemed to me a very good solution. I'm wondering why Debian's policy is so strict about not shipping them. The document doesn't explain the reasons, unfortunately, but there must be some. If we learn what the reasons are and it turns out they are not relevant for us, we might well do as you're suggesting. I found a Python packaging FAQ, the question wasn't on the list. http://wiki.debian.org/Python/FAQ Is anyone onboard a Debian developer and can provide any pointers? Maciej [1] http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-byte_compilation From pfelecan at opencsw.org Mon Jul 29 12:19:27 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 12:19:27 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 09:37:28 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 Peter FELECAN : >> "Maciej (Matchek) Blizi?ski" writes: >>> I did not implement compiling one .py file multiple times for >>> different Python versions. >> >> Do you plan to do it later? > > No, unless we figure out and agree on how to do it. > >>> The main concern for the current CAS is the suitability of pattern >>> matching. Is it good enough to just match a part of the path to know >>> which interpreter to use? It should be good enough for most packages, >>> but I'm not sure what the corner cases / contrived path names could >>> be. >> >> Even if it misses some is not blocking as the code is there. Anyhow, the >> difference between .py and .pyc is not so great (and I don't speak about >> .pyo which seems to me a scam), in my experience and if I understood the >> mechanism of dynamic compilation the overhead is incurred only >> once. This is why supplying the compiled components in the package >> seemed to me a very good solution. > > I'm wondering why Debian's policy is so strict about not shipping > them. The document doesn't explain the reasons, unfortunately, but > there must be some. If we learn what the reasons are and it turns out > they are not relevant for us, we might well do as you're suggesting. I > found a Python packaging FAQ, the question wasn't on the list. > > http://wiki.debian.org/Python/FAQ > > Is anyone onboard a Debian developer and can provide any pointers? I'm not, but using the following search equation: "pyc portability" on Google and after a cursory exploration of the proposed hits on the first page I think that the reason for which Debian imposes this is raised by network shares between systems running different interpreters, version wise. The only thing that I didn't do is to ask on the good mailing list which, by the way, I don't know. Here follows the salient citations and my observations: ** [[http://stackoverflow.com/questions/2263356/are-python-2-5-pyc-files-compatible-with-python-2-6-pyc-files][stackoverflow]] "In general, .pyc files are specific to one Python version (although portable across different machine architectures, as long as they're running the same version); the files carry the information about the relevant Python version in their headers -- so, if you leave the corresponding .py files next to the .pyc ones, the .pyc will be rebuilt every time a different Python version is used to import those modules." ** [[http://docs.python.org/2/tutorial/modules.html#compiled-python-files][tutorial]] 6.1.3. "Compiled" Python files As an important speed-up of the start-up time for short programs that use a lot of standard modules, if a file called spam.pyc exists in the directory where spam.py is found, this is assumed to contain an already-"byte-compiled" version of the module spam. The modification time of the version of spam.py used to create spam.pyc is recorded in spam.pyc, and the .pyc file is ignored if these don't match. Normally, you don't need to do anything to create the spam.pyc file. Whenever spam.py is successfully compiled, an attempt is made to write the compiled version to spam.pyc. It is not an error if this attempt fails; if for any reason the file is not written completely, the resulting spam.pyc file will be recognized as invalid and thus ignored later. The contents of the spam.pyc file are platform independent, so a Python module directory can be shared by machines of different architectures. Some tips for experts: 1. When the Python interpreter is invoked with the -O flag, optimized code is generated and stored in .pyo files. The optimizer currently doesn't help much; it only removes assert statements. When -O is used, all bytecode is optimized; .pyc files are ignored and .py files are compiled to optimized bytecode. 2. Passing two -O flags to the Python interpreter (-OO) will cause the bytecode compiler to perform optimizations that could in some rare cases result in malfunctioning programs. Currently only __doc__ strings are removed from the bytecode, resulting in more compact .pyo files. Since some programs may rely on having these available, you should only use this option if you know what you're doing. 3. A program doesn't run any faster when it is read from a .pyc or .pyo file than when it is read from a .py file; the only thing that's faster about .pyc or .pyo files is the speed with which they are loaded. 4. When a script is run by giving its name on the command line, the bytecode for the script is never written to a .pyc or .pyo file. Thus, the startup time of a script may be reduced by moving most of its code to a module and having a small bootstrap script that imports that module. It is also possible to name a .pyc or .pyo file directly on the command line. 5. It is possible to have a file called spam.pyc (or spam.pyo when -O is used) without a file spam.py for the same module. This can be used to distribute a library of Python code in a form that is moderately hard to reverse engineer. 6. The module compileall can create .pyc files (or .pyo files when -O is used) for all modules in a directory. Item 6 refers to distributing code. ** [[http://www.network-theory.co.uk/docs/pytut/CompiledPythonfiles.html][An Introduction to Python]] Guido van Rossum himself says: "[...]The contents of the `spam.pyc' file are platform independent, so a Python module directory can be shared by machines of different architectures." ** [[http://www.velocityreviews.com/forums/t643545-are-pyc-files-portable.html][forum discussion]] Discussion related on usage of .pyc on different versions of Solaris. The conclusion is that architecture doesn't matter, interpreter matter. ** [[http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-byte_compilation][Debian Python Policy]] The discussion has more to do with interpreter version changes than with architecture. ** [[http://www.ehow.com/info_12214796_pyc-files.html][random]] "Uses for ".PYC" Files Modules that are imported into user scripts get compiled by the interpreter before execution. Because these modules tend to undergo repeated use, the interpreter compiles the module and stores the ".pyc" file in a directory. This way, when a script imports that module, the bytecode version already exists, ready for use. Furthermore, bytecode ".pyc" files are portable across multiple platforms, making pre-compiling Python scripts useful for distributing Python programs across different operating systems." >From my standpoint, here is the exceptional case where Debian is not to be followed. Note that they have a quite similar policy of Emacs Lisp pre-compiled files which is of the same order. What we should do is to supply the compiled files in the package, for each version for which the maintainer thinks that it's adequate. Simpler and leaner, isn't it? -- Peter From maciej at opencsw.org Mon Jul 29 13:27:49 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 12:27:49 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN : > What we should do is to supply the compiled files in the package, for > each version for which the maintainer thinks that it's > adequate. Simpler and leaner, isn't it? You certainly have a point. I talked to folks on the #debian-python channel and asked them about the reasons behind this policy decision. They didn't know of any failure modes, it boiled down to saving mirror disk space and bandwidth. Maciej From pfelecan at opencsw.org Mon Jul 29 14:37:41 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 14:37:41 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 12:27:49 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 Peter FELECAN : >> What we should do is to supply the compiled files in the package, for >> each version for which the maintainer thinks that it's >> adequate. Simpler and leaner, isn't it? > > You certainly have a point. > > I talked to folks on the #debian-python channel and asked them about > the reasons behind this policy decision. They didn't know of any > failure modes, it boiled down to saving mirror disk space and > bandwidth. Yeah. Exactly the arguments when wrangling on similar issues with Philip Brown. Consequently, we should change our ways... On the other hand, I changed the following files in .buildsys/v2, following your proposal: Index: v2/categories/python/category.mk =================================================================== --- v2/categories/python/category.mk (revision 21547) +++ v2/categories/python/category.mk (working copy) @@ -1,5 +1,5 @@ # Add a dependency to CSWpython -_EXTRA_GAR_PKGS += CSWpython +_EXTRA_GAR_PKGS += $(PYTHON_PACKAGE) # For the record, do not include the following line: # _MERGE_EXCLUDE_CATEGORY += .*\.egg-info.* @@ -28,11 +28,11 @@ LICENSE ?= PKG-INFO SPKG_SOURCEURL ?= http://pypi.python.org/pypi/$(NAME) MASTER_SITES ?= $(PYPI_MIRROR) -PACKAGES ?= CSWpy-$(DASHED_NAME) +PACKAGES ?= $(PYTHON_MODULE_PACKAGE_PREFIX)$(DASHED_NAME) # for use in any references by specific recipes so it can be replaced easily # across the tree. this could later be parameterized for use by multiple # versions of python too. -SITE_PACKAGES = $(libdir)/python2.6/site-packages +SITE_PACKAGES = $(PYTHON_SITE_PACKAGES) include gar/gar.mk Index: v2/gar.conf.mk =================================================================== --- v2/gar.conf.mk (revision 21547) +++ v2/gar.conf.mk (working copy) @@ -869,6 +869,31 @@ INSTALL_SCRIPTS ?= $(WORKSRC)/Makefile +# +# Python configuration + +PYTHON_VERSION ?= 2_6 + +PYTHON_EXECUTABLE_2_6 = /opt/csw/bin/python2.6 +PYTHON_EXECUTABLE_2_7 = /opt/csw/bin/python2.7 +PYTHON_EXECUTABLE_3_3 = /opt/csw/bin/python3.3 +PYTHON_EXECUTABLE = $(PYTHON_EXECUTABLE_$(PYTHON_VERSION)) + +PYTHON_PACKAGE_2_6 = CSWpython +PYTHON_PACKAGE_2_7 = CSWpython27 +PYTHON_PACKAGE_3_3 = CSWpython33 +PYTHON_PACKAGE = $(PYTHON_PACKAGE_$(PYTHON_VERSION)) + +PYTHON_SITE_PACKAGES_2_6 = $(libdir)/python2.6/site-packages +PYTHON_SITE_PACKAGES_2_7 = $(libdir)/python2.7/site-packages +PYTHON_SITE_PACKAGES_3_3 = $(libdir)/python3.3/site-packages +PYTHON_SITE_PACKAGES = $(PYTHON_SITE_PACKAGES_$(PYTHON_VERSION)) + +PYTHON_MODULE_PACKAGE_PREFIX_2_6 = CSWpy- +PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy- +PYTHON_MODULE_PACKAGE_PREFIX_3_3 = CSWpy33- +PYTHON_MODULE_PACKAGE_PREFIX = $(PYTHON_MODULE_PACKAGE_PREFIX_$(PYTHON_VERSION)) + # Global environment export PATH PKG_CONFIG_PATH Index: v2/gar.lib.mk =================================================================== --- v2/gar.lib.mk (revision 21547) +++ v2/gar.lib.mk (working copy) @@ -910,7 +910,7 @@ PYBUILD_CMD ?= build build-%/setup.py: @echo " ==> Running setup.py $(PYBUILD_TYPE) in $*" - @( cd $* ; $(BUILD_ENV) python ./setup.py $(PYBUILD_CMD) $(BUILD_ARGS) ) + @( cd $* ; $(BUILD_ENV) $(PYTHON_EXECUTABLE) ./setup.py $(PYBUILD_CMD) $(BUILD_ARGS) ) @$(MAKECOOKIE) #################### CLEAN RULES #################### @@ -988,6 +988,7 @@ test-%/setup.py: @echo " ==> Running setup.py test in $*" @( cd $* ; cd $(OBJDIR) ; $(TEST_ENV) python ./setup.py test $(TEST_ARGS) ) + @( cd $* ; cd $(OBJDIR) ; $(TEST_ENV) $(PYTHON_EXECUTABLE) ./setup.py test $(TEST_ARGS) ) @$(MAKECOOKIE) ################# INSTALL RULES #################### @@ -1035,7 +1036,7 @@ PYINSTALL_CMD ?= install install-%/setup.py: @echo " ==> Running setup.py $(PYINSTALL_CMD) in $*" - @( cd $* ; $(INSTALL_ENV) python ./setup.py $(PYINSTALL_CMD) $(INSTALL_ARGS) ) + @( cd $* ; $(INSTALL_ENV) $(PYTHON_EXECUTABLE) ./setup.py $(PYINSTALL_CMD) $(INSTALL_ARGS) ) @$(MAKECOOKIE) # pkg-config scripts the only omission is using a different prefix for 2.6 and 2.7 packages. Tested it thoroughly and everything seems alright. Do you mind if I commit these changes? This way we have a necessary and sufficient set of changes to support our new toy... -- Peter From maciej at opencsw.org Mon Jul 29 14:54:39 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 13:54:39 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN : > the only omission is using a different prefix for 2.6 and 2.7 packages. > > Tested it thoroughly and everything seems alright. > > Do you mind if I commit these changes? This way we have a necessary and > sufficient set of changes to support our new toy... This line: PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy- ...is fine as long as we don't rebuild the CSWpy- packages with Python 2.7; or more specifically, as long as the packages with the CSWpy- prefix do provide files in the /opt/csw/lib/python/site-packages or /opt/csw/lib/python2.6/site-packages directories. This can be easily encoded into a check in checkpkg. I'm sure you understand the reasoning ? we don't want a regression in 2.6. Maciej From pfelecan at opencsw.org Mon Jul 29 15:27:26 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 15:27:26 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 13:54:39 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 Peter FELECAN : >> the only omission is using a different prefix for 2.6 and 2.7 packages. >> >> Tested it thoroughly and everything seems alright. >> >> Do you mind if I commit these changes? This way we have a necessary and >> sufficient set of changes to support our new toy... > > This line: > > PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy- > > ...is fine as long as we don't rebuild the CSWpy- packages with Python > 2.7; or more specifically, as long as the packages with the CSWpy- > prefix do provide files in the /opt/csw/lib/python/site-packages or > /opt/csw/lib/python2.6/site-packages directories. This can be easily > encoded into a check in checkpkg. I'm sure you understand the > reasoning ? we don't want a regression in 2.6. There are the following cases: 1. The package provides a 2.7 module, i.e., its code use new 2.7 features. In this case it must provide its components in the versioned trees, e.g. /opt/csw/lib/python2.7/site-packages. Consequently, no need for a specific prefix and its dependencies clearly says CSWpython27 2. The package provides a "mixed" module, i.e. the code runs on 2.6 and 2.7 interpreter. In this case it should provide a double pronged installation, i.e., components installed in /opt/csw/python/site-packages. consequently, no need for a specific prefix and its dependency specifies CSWpython *and* CSWpython27. 3. The package provides a 2.6 module (IMHO this is also a 2.7 module). In this case its components should go in /opt/csw/lib/python26 and /opt/csw/python27. Consequently there is no need for a specific prefix. 4. Finally, if, as I suggested, the next transition from unstable to a named catalog defines as an objective to make 2.7 the default interpreter, there is no need for a specific prefix. There are certainly other cases but I think that they can be reduced to a solution which don't need the unnecessary CSWpy27- prefix. I'm willing to falsify the need for each one as a penitence. -- Peter From maciej at opencsw.org Mon Jul 29 16:20:19 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 15:20:19 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN : > "Maciej (Matchek) Blizi?ski" writes: > >> 2013/7/29 Peter FELECAN : >>> the only omission is using a different prefix for 2.6 and 2.7 packages. >>> >>> Tested it thoroughly and everything seems alright. >>> >>> Do you mind if I commit these changes? This way we have a necessary and >>> sufficient set of changes to support our new toy... >> >> This line: >> >> PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy- >> >> ...is fine as long as we don't rebuild the CSWpy- packages with Python >> 2.7; or more specifically, as long as the packages with the CSWpy- >> prefix do provide files in the /opt/csw/lib/python/site-packages or >> /opt/csw/lib/python2.6/site-packages directories. This can be easily >> encoded into a check in checkpkg. I'm sure you understand the >> reasoning ? we don't want a regression in 2.6. > > There are the following cases: > > 1. The package provides a 2.7 module, i.e., its code use new 2.7 > features. In this case it must provide its components in the > versioned trees, > e.g. /opt/csw/lib/python2.7/site-packages. Consequently, no need for > a specific prefix and its dependencies clearly says CSWpython27 One problem with this is the following use case: A user runs Python 2.6 and thinks "I want the foo module. Does OpenCSW provide it?" - runs a quick pkgutil search - "Cool, there's my CSWpy-foo in the catalog, let's install it... let's run import foo... gah, doesn't work! why is this stuff always broken! opencsw is broken!" Can we prevent the above scenario in any way? For example, provide a stub for 2.6 that when you try to import it, it would say: "this package is for Python 2.7 only"? > 2. The package provides a "mixed" module, i.e. the code runs on 2.6 and > 2.7 interpreter. In this case it should provide a double pronged > installation, i.e., components installed in > /opt/csw/python/site-packages. Or instead, let's just provide files in /opt/csw/python2.6/site-packages and /opt/csw/python2.7/site-packages plus their compiled .pyc counterparts, compiled with respective interpreters. Simple to do and effective. Why shouldn't we do this? > consequently, no need for a specific > prefix and its dependency specifies CSWpython *and* CSWpython27. Agreed on the deps. (Also note: if we had CSWpy26- and CSWpy27-, we could be more selective.) > 3. The package provides a 2.6 module (IMHO this is also a 2.7 > module). It surely is, except the 2.6 .pyc files won't work for 2.7. > In this case its components should go in > /opt/csw/lib/python26 and /opt/csw/python27. Consequently there is no > need for a specific prefix. Agreed. > 4. Finally, if, as I suggested, the next transition from unstable to a > named catalog defines as an objective to make 2.7 the default > interpreter, there is no need for a specific prefix. As long as you don't fall into the "let's only support one Python version" trap, agreed. Maciej From pfelecan at opencsw.org Mon Jul 29 16:38:39 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 16:38:39 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 15:20:19 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 Peter FELECAN : >> "Maciej (Matchek) Blizi?ski" writes: >> >>> 2013/7/29 Peter FELECAN : >>>> the only omission is using a different prefix for 2.6 and 2.7 packages. >>>> >>>> Tested it thoroughly and everything seems alright. >>>> >>>> Do you mind if I commit these changes? This way we have a necessary and >>>> sufficient set of changes to support our new toy... >>> >>> This line: >>> >>> PYTHON_MODULE_PACKAGE_PREFIX_2_7 = CSWpy- >>> >>> ...is fine as long as we don't rebuild the CSWpy- packages with Python >>> 2.7; or more specifically, as long as the packages with the CSWpy- >>> prefix do provide files in the /opt/csw/lib/python/site-packages or >>> /opt/csw/lib/python2.6/site-packages directories. This can be easily >>> encoded into a check in checkpkg. I'm sure you understand the >>> reasoning ? we don't want a regression in 2.6. >> >> There are the following cases: >> >> 1. The package provides a 2.7 module, i.e., its code use new 2.7 >> features. In this case it must provide its components in the >> versioned trees, >> e.g. /opt/csw/lib/python2.7/site-packages. Consequently, no need for >> a specific prefix and its dependencies clearly says CSWpython27 > > One problem with this is the following use case: > > A user runs Python 2.6 and thinks "I want the foo module. Does OpenCSW > provide it?" - runs a quick pkgutil search - "Cool, there's my > CSWpy-foo in the catalog, let's install it... let's run import foo... > gah, doesn't work! why is this stuff always broken! opencsw is > broken!" > > Can we prevent the above scenario in any way? For example, provide a > stub for 2.6 that when you try to import it, it would say: "this > package is for Python 2.7 only"? Meh. I don't know. It seems contrived to me. >> 2. The package provides a "mixed" module, i.e. the code runs on 2.6 and >> 2.7 interpreter. In this case it should provide a double pronged >> installation, i.e., components installed in >> /opt/csw/python/site-packages. > > Or instead, let's just provide files in > /opt/csw/python2.6/site-packages and /opt/csw/python2.7/site-packages > plus their compiled .pyc counterparts, compiled with respective > interpreters. Simple to do and effective. Why shouldn't we do this? Indeed. What I thought but not wrote... >> consequently, no need for a specific >> prefix and its dependency specifies CSWpython *and* CSWpython27. > > Agreed on the deps. (Also note: if we had CSWpy26- and CSWpy27-, we > could be more selective.) > >> 3. The package provides a 2.6 module (IMHO this is also a 2.7 >> module). > > It surely is, except the 2.6 .pyc files won't work for 2.7. Are you sure of that? Anyway, the .pyc is rejected and .py is used which incurs just the extra parsing, isn't it? >> In this case its components should go in >> /opt/csw/lib/python26 and /opt/csw/python27. Consequently there is no >> need for a specific prefix. > > Agreed. > >> 4. Finally, if, as I suggested, the next transition from unstable to a >> named catalog defines as an objective to make 2.7 the default >> interpreter, there is no need for a specific prefix. > > As long as you don't fall into the "let's only support one Python > version" trap, agreed. What I propose is eventually to support 2.x and 3.x. Can I commit the changes? -- Peter From maciej at opencsw.org Mon Jul 29 16:59:05 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 15:59:05 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN : >> Can we prevent the above scenario in any way? For example, provide a >> stub for 2.6 that when you try to import it, it would say: "this >> package is for Python 2.7 only"? > > Meh. I don't know. It seems contrived to me. Contrived ? the problem or the solution? The problem seems pretty straightforward to me: you install CSWpy-foo and "import foo" fails. >>> 2. The package provides a "mixed" module, i.e. the code runs on 2.6 and >>> 2.7 interpreter. In this case it should provide a double pronged >>> installation, i.e., components installed in >>> /opt/csw/python/site-packages. >> >> Or instead, let's just provide files in >> /opt/csw/python2.6/site-packages and /opt/csw/python2.7/site-packages >> plus their compiled .pyc counterparts, compiled with respective >> interpreters. Simple to do and effective. Why shouldn't we do this? > > Indeed. What I thought but not wrote... Cool, we have agreement on this one. It should be fairly easy to implement with GAR modulations. I already briefly talked about this with Dago. >>> consequently, no need for a specific >>> prefix and its dependency specifies CSWpython *and* CSWpython27. >> >> Agreed on the deps. (Also note: if we had CSWpy26- and CSWpy27-, we >> could be more selective.) >> >>> 3. The package provides a 2.6 module (IMHO this is also a 2.7 >>> module). >> >> It surely is, except the 2.6 .pyc files won't work for 2.7. > > Are you sure of that? Yes, see here: $ mkdir pyc_test $ cd pyc_test $ echo "print 'foo'" > foo.py $ python2.6 -m foo foo $ python2.6 -m py_compile foo.py $ ls -l foo.pyc -rw-rw-r-- 1 maciej maciej 106 Jul 29 15:44 foo.pyc $ mv foo.py foo_renamed.py $ python2.6 -m foo foo $ python2.7 -m foo /usr/bin/python2.7: No code object available for foo > Anyway, the .pyc is rejected and .py is used which > incurs just the extra parsing, isn't it? Yes. Here's the hierarchy of usefulness: - providing the right .pyc file - is better than not providing a .pyc file at all - which in turn is better than providing the wrong .pyc file We can accept this as a temporary state, but we should work towards the dual 2.6+2.7 support. > What I propose is eventually to support 2.x and 3.x. I don't know if we can talk in terms of 2.x or 3.x. We aim to support 2.6, 2.7 and 3.3. We currently do not support 2.5 (even though 2.5 is a 2.x) nor 3.0 or 3.1 or 3.2. We'll drop the 2.6 at some point when everything works well with 2.7. By the way, do you haven an idea how to deprecate the old 2.6 modules? > Can I commit the changes? Go ahead. I'll add the check to make sure we're not introducing regressions for 2.6. Maciej From pfelecan at opencsw.org Mon Jul 29 17:13:30 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Mon, 29 Jul 2013 17:13:30 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 15:59:05 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 Peter FELECAN : >>> Can we prevent the above scenario in any way? For example, provide a >>> stub for 2.6 that when you try to import it, it would say: "this >>> package is for Python 2.7 only"? >> >> Meh. I don't know. It seems contrived to me. > > Contrived ? the problem or the solution? The problem seems pretty > straightforward to me: you install CSWpy-foo and "import foo" fails. The solution. >>>> 2. The package provides a "mixed" module, i.e. the code runs on 2.6 and >>>> 2.7 interpreter. In this case it should provide a double pronged >>>> installation, i.e., components installed in >>>> /opt/csw/python/site-packages. >>> >>> Or instead, let's just provide files in >>> /opt/csw/python2.6/site-packages and /opt/csw/python2.7/site-packages >>> plus their compiled .pyc counterparts, compiled with respective >>> interpreters. Simple to do and effective. Why shouldn't we do this? >> >> Indeed. What I thought but not wrote... > > Cool, we have agreement on this one. It should be fairly easy to > implement with GAR modulations. I already briefly talked about this > with Dago. Good to talk with Dago about this. He can bring his salt. >>>> consequently, no need for a specific >>>> prefix and its dependency specifies CSWpython *and* CSWpython27. >>> >>> Agreed on the deps. (Also note: if we had CSWpy26- and CSWpy27-, we >>> could be more selective.) >>> >>>> 3. The package provides a 2.6 module (IMHO this is also a 2.7 >>>> module). >>> >>> It surely is, except the 2.6 .pyc files won't work for 2.7. >> >> Are you sure of that? > > Yes, see here: > > $ mkdir pyc_test > $ cd pyc_test > $ echo "print 'foo'" > foo.py > $ python2.6 -m foo > foo > $ python2.6 -m py_compile foo.py > $ ls -l foo.pyc > -rw-rw-r-- 1 maciej maciej 106 Jul 29 15:44 foo.pyc > $ mv foo.py foo_renamed.py > $ python2.6 -m foo > foo > $ python2.7 -m foo > /usr/bin/python2.7: No code object available for foo Right. But I wa writing about the case when .py exists and .pyc is from a earlier version. That works, isn't it? >> Anyway, the .pyc is rejected and .py is used which >> incurs just the extra parsing, isn't it? > > Yes. Here's the hierarchy of usefulness: > > - providing the right .pyc file > - is better than not providing a .pyc file at all > - which in turn is better than providing the wrong .pyc file > > We can accept this as a temporary state, but we should work towards > the dual 2.6+2.7 support. > >> What I propose is eventually to support 2.x and 3.x. > > I don't know if we can talk in terms of 2.x or 3.x. We aim to support > 2.6, 2.7 and 3.3. We currently do not support 2.5 (even though 2.5 is > a 2.x) nor 3.0 or 3.1 or 3.2. We'll drop the 2.6 at some point when > everything works well with 2.7. Nah. When I write "support 2.x and 3.x" I mean, support the latest version of 2 and 3. in Our case 2.7 and 3.3. > By the way, do you haven an idea how to deprecate the old 2.6 modules? > >> Can I commit the changes? > > Go ahead. I'll add the check to make sure we're not introducing > regressions for 2.6. Thanks. -- Peter From dam at opencsw.org Mon Jul 29 20:24:37 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Mon, 29 Jul 2013 20:24:37 +0200 Subject: [csw-maintainers] upgrade sourceforge? In-Reply-To: References: Message-ID: <142488C7-B3BF-47F3-B576-6F4ED416D346@opencsw.org> Hi Rupert, Am 28.07.2013 um 12:14 schrieb rupert THURNER : > committing to sourceforge gives an upgrade warning. do you plan to upgrade? Not yet, I have set "gar" on the delay-list so we don't get automatic migration because Allura still misses some features on Trac. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From maciej at opencsw.org Tue Jul 30 00:33:30 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 23:33:30 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN : >>> Can I commit the changes? >> >> Go ahead. I'll add the check to make sure we're not introducing >> regressions for 2.6. > > Thanks. Done. Making sure that if a package is named CSWpy-*, it must provide files reachable by Python 2.6: http://sourceforge.net/apps/trac/gar/changeset/21575 ...and making sure that a package containing files for a specific Python version depends on that specific version. http://sourceforge.net/apps/trac/gar/changeset/21577 Maciej From maciej at opencsw.org Tue Jul 30 00:48:23 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Mon, 29 Jul 2013 23:48:23 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: One more issue to solve before we can deprecate Python 2.6: we need to support Python on all the platforms on which we're building. Our current Solaris 9 builds are functioning thanks to an old Python 2.6 build on Solaris 9. If we want to get rid of Python 2.6, we need to make 2.7 work on Solaris 9, or stop building on Solaris 9. Maciej From pfelecan at opencsw.org Tue Jul 30 09:39:44 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 09:39:44 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 23:48:23 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > One more issue to solve before we can deprecate Python 2.6: we need to > support Python on all the platforms on which we're building. Our > current Solaris 9 builds are functioning thanks to an old Python 2.6 > build on Solaris 9. If we want to get rid of Python 2.6, we need to > make 2.7 work on Solaris 9, or stop building on Solaris 9. Packaging for Solaris 9 was deprecated last year and it was an activity based on voluntary work. Consequently, this is a non issue. -- Peter From pfelecan at opencsw.org Tue Jul 30 09:42:43 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 09:42:43 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Mon, 29 Jul 2013 23:33:30 +0100") References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/29 Peter FELECAN : >>>> Can I commit the changes? >>> >>> Go ahead. I'll add the check to make sure we're not introducing >>> regressions for 2.6. >> >> Thanks. > > Done. > > Making sure that if a package is named CSWpy-*, it must provide files > reachable by Python 2.6: > http://sourceforge.net/apps/trac/gar/changeset/21575 What if the packaged project supports only 2.7? An override exist for this? What if the maintainer is not willing to provide a 2.6 version of his package? Especially in the case where it's a new package, never released before for 2.6. This will hinder the transition to 2.7. -- Peter From maciej at opencsw.org Tue Jul 30 09:50:32 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 30 Jul 2013 08:50:32 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/30 Peter FELECAN > > > Making sure that if a package is named CSWpy-*, it must provide files > > reachable by Python 2.6: > > http://sourceforge.net/apps/trac/gar/changeset/21575 > > What if the packaged project supports only 2.7? Unlikely, but a theoretical possibility. > An override exist for > this? Sure thing, you can have an override for anything. All error tags are the same, and they can all be overriden just the same. > What if the maintainer is not willing to provide a 2.6 version of his > package? Especially in the case where it's a new package, never released > before for 2.6. > > This will hinder the transition to 2.7. If you know what you're doing, you just add an override. The idea behind the check is to prevent maintainers from unintentionally missing the 2.6 files for a module. From pfelecan at opencsw.org Tue Jul 30 09:55:40 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 09:55:40 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Tue, 30 Jul 2013 08:50:32 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/30 Peter FELECAN >> >> > Making sure that if a package is named CSWpy-*, it must provide files >> > reachable by Python 2.6: >> > http://sourceforge.net/apps/trac/gar/changeset/21575 >> >> What if the packaged project supports only 2.7? > > Unlikely, but a theoretical possibility. All this 2.7 thread started when I tried to package a project supported only by a Python interpreter 2.7 or greater. So it's more than theoretical. -- Peter From maciej at opencsw.org Tue Jul 30 09:56:31 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 30 Jul 2013 08:56:31 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/30 Peter FELECAN : > Packaging for Solaris 9 was deprecated last year and it was an activity > based on voluntary work. Consequently, this is a non issue. Yes, but it changes the game for the volunteers. Previously: - If you want to volunteer to build a Solaris 9 package, you simply use the build tools as usual and you resolve any potential Solaris 9 specific problems. After: - if you want to volunteer to build a Solaris 9 package, you have to rework all the tooling and release them for Solaris 9, but even releasing doesn't work so you have a chicken and egg problem and you need low-level access to the buildfarm to do anything. What I'm saying is that it will make it practically impossible for people to volunteer. To say that you can just volunteer to build a Solaris 9 package if you wish, would no longer be honest of us. Maciej From maciej at opencsw.org Tue Jul 30 09:57:26 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 30 Jul 2013 08:57:26 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/30 Peter FELECAN : > All this 2.7 thread started when I tried to package a project supported > only by a Python interpreter 2.7 or greater. So it's more than > theoretical. It's an application, not a library. Most libraries try not to break compatibility with 2.5 and 2.6. From maciej at opencsw.org Tue Jul 30 10:08:53 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 30 Jul 2013 09:08:53 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: By the way, if anyone thinks that I'm hindering progress, please let me know! In my mind, I'm just making sure that we have a good migration path for our users. I strongly support a plan of transitioning to 2.7 and killing off 2.6. I just think that we cannot jump from 2.6 to 2.7 without a period of time when both interpreters work roughly equally well. This will allow our users to migrate. Then we kill 2.6. Maciej From pfelecan at opencsw.org Tue Jul 30 10:11:23 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 10:11:23 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Tue, 30 Jul 2013 08:56:31 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/30 Peter FELECAN : >> Packaging for Solaris 9 was deprecated last year and it was an activity >> based on voluntary work. Consequently, this is a non issue. > > Yes, but it changes the game for the volunteers. Previously: > > - If you want to volunteer to build a Solaris 9 package, you simply > use the build tools as usual and you resolve any potential Solaris 9 > specific problems. > > After: > > - if you want to volunteer to build a Solaris 9 package, you have to > rework all the tooling and release them for Solaris 9, but even > releasing doesn't work so you have a chicken and egg problem and you > need low-level access to the buildfarm to do anything. > > What I'm saying is that it will make it practically impossible for > people to volunteer. To say that you can just volunteer to build a > Solaris 9 package if you wish, would no longer be honest of us. Maybe the time has come to stop event that. -- Peter From pfelecan at opencsw.org Tue Jul 30 10:12:46 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 10:12:46 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Tue, 30 Jul 2013 08:57:26 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > 2013/7/30 Peter FELECAN : >> All this 2.7 thread started when I tried to package a project supported >> only by a Python interpreter 2.7 or greater. So it's more than >> theoretical. > > It's an application, not a library. Most libraries try not to break > compatibility with 2.5 and 2.6. You're right. But that doesn't preclude me to conjecture the existence of that kind of modules. -- Peter From maciej at opencsw.org Tue Jul 30 10:21:04 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 30 Jul 2013 09:21:04 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: 2013/7/30 Peter FELECAN : > You're right. But that doesn't preclude me to conjecture the existence > of that kind of modules. Sure. That's what overrides are for ? meaning, for situations that are more on the exception side rather than the rule. Maciej From pfelecan at opencsw.org Tue Jul 30 10:28:54 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 10:28:54 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Tue, 30 Jul 2013 09:08:53 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > By the way, if anyone thinks that I'm hindering progress, please let > me know! In my mind, I'm just making sure that we have a good > migration path for our users. I strongly support a plan of > transitioning to 2.7 and killing off 2.6. I just think that we cannot > jump from 2.6 to 2.7 without a period of time when both interpreters > work roughly equally well. This will allow our users to migrate. Then > we kill 2.6. >From my stand point no, on the contrary. -- Peter From maciej at opencsw.org Tue Jul 30 16:08:15 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Tue, 30 Jul 2013 15:08:15 +0100 Subject: [csw-maintainers] Fwd: [serf-dev] building on solaris In-Reply-To: References: <27657294-cd9a-46b4-818f-3d0ba6a88e0e@googlegroups.com> Message-ID: Maybe we can get Greg a buildfarm account? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dam at opencsw.org Tue Jul 30 16:09:14 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 30 Jul 2013 16:09:14 +0200 Subject: [csw-maintainers] Fwd: [serf-dev] building on solaris In-Reply-To: References: <27657294-cd9a-46b4-818f-3d0ba6a88e0e@googlegroups.com> Message-ID: <19377168-EC8B-4F97-94FF-58500B635A8D@opencsw.org> Hi Maciej, Am 30.07.2013 um 16:08 schrieb Maciej (Matchek) Blizi?ski : > Maybe we can get Greg a buildfarm account? Already done yesterday :-) Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From pfelecan at opencsw.org Tue Jul 30 20:20:08 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 20:20:08 +0200 Subject: [csw-maintainers] excluding components from a package Message-ID: I have an issue as follows: - package pp has in its PKGFILES a set of files fp, explicitly defined - package pd has in its PKGFILES a set of file fd, implicitly defined through PKGFILES_DEVEL macro - the packages are defined in pp, pd order - the fp set is include in the fd set How can I exclude from fd the content of fp, i.e. complement it? Is there a known way or should I resort to my magic wand? TIA for any answer -- Peter From dam at opencsw.org Tue Jul 30 22:21:01 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 30 Jul 2013 22:21:01 +0200 Subject: [csw-maintainers] excluding components from a package In-Reply-To: References: Message-ID: <23A4D316-4B3B-40C6-8720-20851657A186@opencsw.org> Hi Peter, Am 30.07.2013 um 20:20 schrieb Peter FELECAN : > I have an issue as follows: > > - package pp has in its PKGFILES a set of files fp, explicitly defined > - package pd has in its PKGFILES a set of file fd, implicitly defined > through PKGFILES_DEVEL macro > - the packages are defined in pp, pd order > - the fp set is include in the fd set > > How can I exclude from fd the content of fp, i.e. complement it? > > Is there a known way or should I resort to my magic wand? No magic wand needed :-) You can either use the respective expression from PKGFILES_DEVEL explicitly or reset the components from PKGFILES_DEVEL to empty which belong to fp as in the composite variable defined here: http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.pkg.mk#L383 like PKGFILES_DEVEL_LIBTOOL = to exclude *.la files from fd. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From pfelecan at opencsw.org Tue Jul 30 22:31:36 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Tue, 30 Jul 2013 22:31:36 +0200 Subject: [csw-maintainers] excluding components from a package In-Reply-To: <23A4D316-4B3B-40C6-8720-20851657A186@opencsw.org> (Dagobert Michelsen's message of "Tue, 30 Jul 2013 22:21:01 +0200") References: <23A4D316-4B3B-40C6-8720-20851657A186@opencsw.org> Message-ID: Dagobert Michelsen writes: > Hi Peter, > > Am 30.07.2013 um 20:20 schrieb Peter FELECAN : >> I have an issue as follows: >> >> - package pp has in its PKGFILES a set of files fp, explicitly defined >> - package pd has in its PKGFILES a set of file fd, implicitly defined >> through PKGFILES_DEVEL macro >> - the packages are defined in pp, pd order >> - the fp set is include in the fd set >> >> How can I exclude from fd the content of fp, i.e. complement it? >> >> Is there a known way or should I resort to my magic wand? > > > No magic wand needed :-) > > You can either use the respective expression from PKGFILES_DEVEL explicitly > or reset the components from PKGFILES_DEVEL to empty which belong to fp as > in the composite variable defined here: > http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.pkg.mk#L383 > like > PKGFILES_DEVEL_LIBTOOL = > to exclude *.la files from fd. I'm not sure that this will do as fp is composed of include files, sub-tree of $(includedir), and fd contains all the relevant files from $(includedir). This is why I thought about the magic wand. In fact the question is how to exclude from pd some sub-trees. -- Peter From dam at opencsw.org Tue Jul 30 22:33:39 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 30 Jul 2013 22:33:39 +0200 Subject: [csw-maintainers] excluding components from a package In-Reply-To: References: <23A4D316-4B3B-40C6-8720-20851657A186@opencsw.org> Message-ID: <0E16F13D-EC4F-419E-AC11-F41C75E158C3@opencsw.org> Hi Peter, Am 30.07.2013 um 22:31 schrieb Peter FELECAN : > Dagobert Michelsen writes: > >> Hi Peter, >> >> Am 30.07.2013 um 20:20 schrieb Peter FELECAN : >>> I have an issue as follows: >>> >>> - package pp has in its PKGFILES a set of files fp, explicitly defined >>> - package pd has in its PKGFILES a set of file fd, implicitly defined >>> through PKGFILES_DEVEL macro >>> - the packages are defined in pp, pd order >>> - the fp set is include in the fd set >>> >>> How can I exclude from fd the content of fp, i.e. complement it? >>> >>> Is there a known way or should I resort to my magic wand? >> >> >> No magic wand needed :-) >> >> You can either use the respective expression from PKGFILES_DEVEL explicitly >> or reset the components from PKGFILES_DEVEL to empty which belong to fp as >> in the composite variable defined here: >> http://sourceforge.net/apps/trac/gar/browser/csw/mgar/gar/v2/gar.pkg.mk#L383 >> like >> PKGFILES_DEVEL_LIBTOOL = >> to exclude *.la files from fd. > > I'm not sure that this will do as fp is composed of include files, > sub-tree of $(includedir), and fd contains all the relevant files from > $(includedir). This is why I thought about the magic wand. In fact the > question is how to exclude from pd some sub-trees. I see. Unfortunately there is no easy way for that, you must restrict the included pathes for fd to be disjunct with fp manually. Sorry. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From maciej at opencsw.org Wed Jul 31 08:03:10 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Wed, 31 Jul 2013 07:03:10 +0100 Subject: [csw-maintainers] Modulations in GAR: How to control merging? Message-ID: Here's how I edited the a Python module build recipe: Index: lang-python/webpy/trunk/Makefile =================================================================== --- lang-python/webpy/trunk/Makefile (revision 20848) +++ lang-python/webpy/trunk/Makefile (working copy) (...) +EXTRA_MODULATORS = PYTHON_VERSION +MODULATIONS_PYTHON_VERSION = 2_6 2_7 +MERGE_SCRIPTS = copy-all include gar/category.mk If I set MERGE_SCRIPTS to copy-all (confirmed by mgar modenv showing "Merge Scripts: copy-all") it should merge everything, isn't that so? But after the merge phase, the pkgroot directory is empty. I verified that the install directories do contain the files they're supposed to. What am I missing? Maciej From maciej at opencsw.org Wed Jul 31 09:05:45 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Wed, 31 Jul 2013 08:05:45 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: I've built a proof of concept package with a module for two Python versions. http://buildfarm.opencsw.org/pkgdb/srv4/0d104a066cc756f822da96867d760a6f/ You can execute the command listed at the bottom of the page to examine the metadata such as the list of files. I'm talking to Dago about how to best add category-level modulations. When we're done with that, we'll mass-rebuild all the Python modules with the new setting. I'm also trying to get Python 2.7 to work on Solaris 9. When running the test suite, test_threading is hanging, and it looks like a bug to me. I've filed http://bugs.python.org/issue18605 to see what does the upstream say. Maciej From pfelecan at opencsw.org Wed Jul 31 09:56:04 2013 From: pfelecan at opencsw.org (Peter FELECAN) Date: Wed, 31 Jul 2013 09:56:04 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: ("Maciej (Matchek) =?utf-8?Q?Blizi=C5=84ski=22's?= message of "Wed, 31 Jul 2013 08:05:45 +0100") References: Message-ID: "Maciej (Matchek) Blizi?ski" writes: > I've built a proof of concept package with a module for two Python versions. > > http://buildfarm.opencsw.org/pkgdb/srv4/0d104a066cc756f822da96867d760a6f/ > > You can execute the command listed at the bottom of the page to > examine the metadata such as the list of files. > > I'm talking to Dago about how to best add category-level modulations. > When we're done with that, we'll mass-rebuild all the Python modules > with the new setting. Great! > I'm also trying to get Python 2.7 to work on Solaris 9. When running > the test suite, test_threading is hanging, and it looks like a bug to > me. I've filed http://bugs.python.org/issue18605 to see what does the > upstream say. As you wish but IMHO you're wasting your time. -- Peter From maciej at opencsw.org Wed Jul 31 09:57:30 2013 From: maciej at opencsw.org (=?UTF-8?Q?Maciej_=28Matchek=29_Blizi=C5=84ski?=) Date: Wed, 31 Jul 2013 08:57:30 +0100 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: <20130725182359.GA20759@cotton.home.blizinski.pl> Message-ID: 2013/7/29 Peter FELECAN > > 2. The package provides a "mixed" module, i.e. the code runs on 2.6 and > 2.7 interpreter. In this case it should provide a double pronged > installation, i.e., components installed in > /opt/csw/python/site-packages. consequently, no need for a specific > prefix and its dependency specifies CSWpython *and* CSWpython27. Here's a radical idea: Let's make the module packages not depend on any Python interpreter. Pros: We won't force people to pull in Python 2.6 if they only want Python 2.7 and vice versa. Cons: Someone might rely on the dependency and expect to install the interpreter by only issuing a command to install a module. Thoughts? Maciej From dam at opencsw.org Wed Jul 31 10:00:10 2013 From: dam at opencsw.org (Dagobert Michelsen) Date: Wed, 31 Jul 2013 10:00:10 +0200 Subject: [csw-maintainers] Python 2.7 In-Reply-To: References: Message-ID: <86604CBF-48B3-4F6C-ACE4-51A71CBD6568@opencsw.org> Hi Maciej, Am 31.07.2013 um 09:05 schrieb Maciej (Matchek) Blizi?ski : > I've built a proof of concept package with a module for two Python versions. > > http://buildfarm.opencsw.org/pkgdb/srv4/0d104a066cc756f822da96867d760a6f/ > > You can execute the command listed at the bottom of the page to > examine the metadata such as the list of files. > > I'm talking to Dago about how to best add category-level modulations. > When we're done with that, we'll mass-rebuild all the Python modules > with the new setting. I followed the discussion on cross-version modules and the more I think about it the more I think it would be better to clearly separate modules for different Python versions. If you already built them with modulations I don't see the point in putting them all in one package instead of having the old (2.6) CSWpy- and the new CSWpy27- and CSWpy33- modules. The only possible benefit I see is that people who are using 2.6 can pkgutil update and switch to 2.7 after all has been rebuilt. But that can be achieved with a online shellscript installing CSWpy27- for all CSWpy- modules. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896