Patchwork [Angstrom-devel] Python library hashlib is missig hash-code.

Op 15 apr. 2012, om 16:35 heeft Koen Kooi het volgende geschreven:

> My theory is that this is caused by the openssl /usr changes, can anyone confirm or deny that?

I dug into this. The breakage was recorded by buildhistory:

==========================================================================

And look at that, an openssl patch in between those dates:

==========================================================================
From 01ea85f7f6c53c66c76d6f832518b28bf06ec072 Mon Sep 17 00:00:00 2001
From: Andrei Gherzan <andrei@gherzan.ro>
Date: Tue, 21 Feb 2012 23:03:55 +0000
Subject: openssl: Move libcrypto to base_libdir

This fix is for dhclient. It needs libcrypto at runtime and if
libcrypto is in libdir, it's path can be inaccessible on systems
where /usr is on nfs for example or dhclient is needed before
/usr is mounted.

Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
==========================================================================

I'm going to band-aid it so we can get a working python hashlib module in the 'denzil' branch, hopefully someone will come up with a proper fix soon.

regards,

Koen


> 
> Begin doorgestuurd bericht:
> 
>> Van: Mattias Millbro <mmillbro@kth.se>
>> Onderwerp: [Angstrom-devel] Python library hashlib is missig hash-code.
>> Datum: 15 april 2012 14:38:56 GMT+02:00
>> Aan: angstrom-distro-devel@linuxtogo.org
>> Antwoord aan: Discussion of the angstrom distribution development <angstrom-distro-devel@linuxtogo.org>
>> 
>> Hi everybody,
>> 
>> as a new member of this mailing list I'm not familiar to the proper procedure of this kids of questions/bug reports. But I'll give it a try and please let me know if any necessary information is missing.
>> 
>> I'm running Angstrom on a Beaglebone using the Angstrom-systemd-image-eglibc-ipk-v2012.01-core-beaglebone.rootfs.tar.bz2 image from http://www.angstrom-distribution.org/demo/beaglebone/. I installed python and pyuthon-modules using opkg.
>> 
>> I recently made a completely fresh install and discovered that the python library hashlib does no longer work properly. It does not seem to find code for sha hashes even tho python.org states
>> 
>> "Included are the FIPS secure hash algorithms SHA1, SHA224, SHA256, SHA384, and SHA512 (defined in FIPS 180-2) as well as RSA’s MD5 algorithm (defined in Internet RFC 1321).".
>> 
>> All related packages seems to be up to date and I have spent several days googling and installing/reinstalling various packages without any success.
>> 
>> This is what happens when importing hashlib:
>> 
>> root@beaglebone:~# python
>> Python 2.7.2 (default, Apr 13 2012, 21:18:55)
>> [GCC 4.5.4 20120305 (prerelease)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import hashlib
>> ERROR:root:code for hash sha1 was not found.
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/hashlib.py", line 139, in <module>
>> globals()[__func_name] = __get_hash(__func_name)
>> File "/usr/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
>> raise ValueError('unsupported hash type %s' % name)
>> ValueError: unsupported hash type sha1
>> ERROR:root:code for hash sha224 was not found.
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/hashlib.py", line 139, in <module>
>> globals()[__func_name] = __get_hash(__func_name)
>> File "/usr/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
>> raise ValueError('unsupported hash type %s' % name)
>> ValueError: unsupported hash type sha224
>> ERROR:root:code for hash sha256 was not found.
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/hashlib.py", line 139, in <module>
>> globals()[__func_name] = __get_hash(__func_name)
>> File "/usr/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
>> raise ValueError('unsupported hash type %s' % name)
>> ValueError: unsupported hash type sha256
>> ERROR:root:code for hash sha384 was not found.
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/hashlib.py", line 139, in <module>
>> globals()[__func_name] = __get_hash(__func_name)
>> File "/usr/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
>> raise ValueError('unsupported hash type %s' % name)
>> ValueError: unsupported hash type sha384
>> ERROR:root:code for hash sha512 was not found.
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/hashlib.py", line 139, in <module>
>> globals()[__func_name] = __get_hash(__func_name)
>> File "/usr/lib/python2.7/hashlib.py", line 91, in __get_builtin_constructor
>> raise ValueError('unsupported hash type %s' % name)
>> ValueError: unsupported hash type sha512
>>>>> 
>> 
>> Here is some additional information that might be useful:
>> 
>> root@beaglebone:~# opkg list-installed | grep python
>> libpython2.7-1.0 - 2.7.2-r2.11
>> python-2to3 - 2.7.2-r2.11
>> python-audio - 2.7.2-r2.11
>> python-bsddb - 2.7.2-r2.11
>> python-codecs - 2.7.2-r2.11
>> python-compile - 2.7.2-r2.11
>> python-compiler - 2.7.2-r2.11
>> python-compression - 2.7.2-r2.11
>> python-core - 2.7.2-r2.11
>> python-crypt - 2.7.2-r2.11
>> python-ctypes - 2.7.2-r2.11
>> python-curses - 2.7.2-r2.11
>> python-datetime - 2.7.2-r2.11
>> python-db - 2.7.2-r2.11
>> python-debugger - 2.7.2-r2.11
>> python-difflib - 2.7.2-r2.11
>> python-distutils - 2.7.2-r2.11
>> python-doctest - 2.7.2-r2.11
>> python-elementtree - 2.7.2-r2.11
>> python-email - 2.7.2-r2.11
>> python-fcntl - 2.7.2-r2.11
>> python-gdbm - 2.7.2-r2.11
>> python-hotshot - 2.7.2-r2.11
>> python-html - 2.7.2-r2.11
>> python-idle - 2.7.2-r2.11
>> python-image - 2.7.2-r2.11
>> python-io - 2.7.2-r2.11
>> python-json - 2.7.2-r2.11
>> python-lang - 2.7.2-r2.11
>> python-logging - 2.7.2-r2.11
>> python-mailbox - 2.7.2-r2.11
>> python-math - 2.7.2-r2.11
>> python-mime - 2.7.2-r2.11
>> python-mmap - 2.7.2-r2.11
>> python-modules - 2.7.2-r2.11
>> python-multiprocessing - 2.7.2-r2.11
>> python-netclient - 2.7.2-r2.11
>> python-netserver - 2.7.2-r2.11
>> python-numbers - 2.7.2-r2.11
>> python-pickle - 2.7.2-r2.11
>> python-pkgutil - 2.7.2-r2.11
>> python-pprint - 2.7.2-r2.11
>> python-profile - 2.7.2-r2.11
>> python-pydoc - 2.7.2-r2.11
>> python-re - 2.7.2-r2.11
>> python-readline - 2.7.2-r2.11
>> python-resource - 2.7.2-r2.11
>> python-robotparser - 2.7.2-r2.11
>> python-shell - 2.7.2-r2.11
>> python-smtpd - 2.7.2-r2.11
>> python-sqlite3 - 2.7.2-r2.11
>> python-sqlite3-tests - 2.7.2-r2.11
>> python-stringold - 2.7.2-r2.11
>> python-subprocess - 2.7.2-r2.11
>> python-syslog - 2.7.2-r2.11
>> python-terminal - 2.7.2-r2.11
>> python-tests - 2.7.2-r2.11
>> python-textutils - 2.7.2-r2.11
>> python-threading - 2.7.2-r2.11
>> python-tkinter - 2.7.2-r2.11
>> python-unittest - 2.7.2-r2.11
>> python-unixadmin - 2.7.2-r2.11
>> python-xml - 2.7.2-r2.11
>> python-xmlrpc - 2.7.2-r2.11
>> python-zlib - 2.7.2-r2.11
>> 
>> root@beaglebone:~# opkg list-installed | grep openssl
>> openssl - 1.0.0h-r15.1
>> 
>> I'm grateful for any kind of help solving this issue.
>> 
>> Best Regards
>> / Mattias
>> 
>> _______________________________________________
>> Angstrom-distro-devel mailing list
>> Angstrom-distro-devel@linuxtogo.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/angstrom-distro-devel
>

Op 23 apr. 2012, om 10:15 heeft Koen Kooi het volgende geschreven:

> Op 15 apr. 2012, om 16:35 heeft Koen Kooi het volgende geschreven:
> 
>> My theory is that this is caused by the openssl /usr changes, can anyone confirm or deny that?
> 
> I dug into this. The breakage was recorded by buildhistory:

[..]

> I'm going to band-aid it so we can get a working python hashlib module in the 'denzil' branch, hopefully someone will come up with a proper fix soon.

After spending some more time on this and getting a ton of help from RP:

Python can't link to anything in ${base_libdir} due to a combination of python having a stupid buildsystem and the patches we apply to make it less stupid. The only fix I found to get hashlib and ssl to work in python is to revert the patch to openssl that moves libcrypto around.

I really need python unbroken in the release branch, so I see a few scenarios:

1) revert openssl patch, seperate /usr doesn't work properly anyway in this release
2) fix python
3) revert openssl patch, statically link libcrypto into dhclient.
4) bbappend openssl in meta-angstrom to revert the openssl patch

There are probably more scenarios, but I'm going to have lunch before thinking some more about it. My question to all of you:

	How do you want to fix this for denzil and how do you want to fix this for master?

And to be clear: I'm mean getting things into the denzil branch of OE-core, not getting it into the release, it's too late for that.

regards,

koen

On Mon, 2012-04-23 at 15:08 +0200, Koen Kooi wrote:
> Op 23 apr. 2012, om 10:15 heeft Koen Kooi het volgende geschreven:
> 
> > Op 15 apr. 2012, om 16:35 heeft Koen Kooi het volgende geschreven:
> > 
> >> My theory is that this is caused by the openssl /usr changes, can
> anyone confirm or deny that?
> > 
> > I dug into this. The breakage was recorded by buildhistory:
> 
> [..]
> 
> > I'm going to band-aid it so we can get a working python hashlib
> module in the 'denzil' branch, hopefully someone will come up with a
> proper fix soon.
> 
> After spending some more time on this and getting a ton of help from
> RP:
> 
> Python can't link to anything in ${base_libdir} due to a combination
> of python having a stupid buildsystem and the patches we apply to make
> it less stupid. The only fix I found to get hashlib and ssl to work in
> python is to revert the patch to openssl that moves libcrypto around.
> 
> I really need python unbroken in the release branch, so I see a few
> scenarios:
> 
> 1) revert openssl patch, seperate /usr doesn't work properly anyway in
> this release
> 2) fix python
> 3) revert openssl patch, statically link libcrypto into dhclient.
> 4) bbappend openssl in meta-angstrom to revert the openssl patch
> 
> There are probably more scenarios, but I'm going to have lunch before
> thinking some more about it. My question to all of you:
> 
> 	How do you want to fix this for denzil and how do you want to fix
> this for master?
> 
> And to be clear: I'm mean getting things into the denzil branch of
> OE-core, not getting it into the release, it's too late for that.

I've managed to squeeze a fix for this in, I posted the patches last
night. It wasn't just the ssl piece that was broken, there were a number
of problems in the python recipe causing various modules to fail to
build (such as curses and readline). Some of these issues had been
present for much longer than the openssl change.

Its worth noting the openssl change was not the problem, it was python
passing native paths to a cross-gcc which then understandably got
confused.

At this point the release is pretty much done, assuming a favourable QA
report.

Cheers,

Richard