Patchwork libxml2: explicitly disable zlib for native builds

login
register
mail settings
Submitter Mario Domenech Goulart
Date Aug. 14, 2014, 8:20 p.m.
Message ID <1408047639-18852-1-git-send-email-mario@ossystems.com.br>
Download mbox | patch
Permalink /patch/78289/
State New
Headers show

Comments

Mario Domenech Goulart - Aug. 14, 2014, 8:20 p.m.
Without this change, meta-openembedded's php recipe (as of 45e62fb8 --
"php 5.4.14: use pkg-config for libxml2 detection") breaks with this
error on my system:

   tmp/sysroots/x86_64-linux/usr/lib/libxml2.so: undefined reference
   to `gzopen64@ZLIB_1.2.3.3'

It seems that the host's zlib is leaking into libxml2's build.

Signed-off-by: Mario Domenech Goulart <mario@ossystems.com.br>
---
 meta/recipes-core/libxml/libxml2.inc |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Richard Purdie - Aug. 21, 2014, 7:50 a.m.
On Thu, 2014-08-14 at 17:20 -0300, Mario Domenech Goulart wrote:
> Without this change, meta-openembedded's php recipe (as of 45e62fb8 --
> "php 5.4.14: use pkg-config for libxml2 detection") breaks with this
> error on my system:
> 
>    tmp/sysroots/x86_64-linux/usr/lib/libxml2.so: undefined reference
>    to `gzopen64@ZLIB_1.2.3.3'
> 
> It seems that the host's zlib is leaking into libxml2's build.

It turns out we need a zlib dependency here, else createrepo fails to
deal with the compressed index files in the rpm repos. With rpm enabled,
you can reproduce with:

"bitbake package-index"

Unfortunately this error causes a cascade of other problems and caused
several people hours of work trying to figure out why the autobuilder
was "exploding".

Basically, the error above triggers a binary character to be emitted
into the logs. The error reporting tool chokes on this since JSON can't
handle the character causing an exception deep in the event handling
code.

That exception is badly handled and causes bitbake to enter an infinite
loop, spewing traceback data to the console. That in turn creates
gigabytes of log files and the autobuilder falls over.

So if anyone wonders why we haven't had patches merging, this is why.
We're now quite behind with the queue.

Obviously there are fixes to make in multiple parts of the system to
stop this kind of error escalating so badly again.

Cheers,

Richard
Otavio Salvador - Aug. 21, 2014, noon
Hello Richard,

On Thu, Aug 21, 2014 at 4:50 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2014-08-14 at 17:20 -0300, Mario Domenech Goulart wrote:
>> Without this change, meta-openembedded's php recipe (as of 45e62fb8 --
>> "php 5.4.14: use pkg-config for libxml2 detection") breaks with this
>> error on my system:
>>
>>    tmp/sysroots/x86_64-linux/usr/lib/libxml2.so: undefined reference
>>    to `gzopen64@ZLIB_1.2.3.3'
>>
>> It seems that the host's zlib is leaking into libxml2's build.
>
> It turns out we need a zlib dependency here, else createrepo fails to
> deal with the compressed index files in the rpm repos. With rpm enabled,
> you can reproduce with:
>
> "bitbake package-index"
>
> Unfortunately this error causes a cascade of other problems and caused
> several people hours of work trying to figure out why the autobuilder
> was "exploding".
>
> Basically, the error above triggers a binary character to be emitted
> into the logs. The error reporting tool chokes on this since JSON can't
> handle the character causing an exception deep in the event handling
> code.
>
> That exception is badly handled and causes bitbake to enter an infinite
> loop, spewing traceback data to the console. That in turn creates
> gigabytes of log files and the autobuilder falls over.
>
> So if anyone wonders why we haven't had patches merging, this is why.
> We're now quite behind with the queue.
>
> Obviously there are fixes to make in multiple parts of the system to
> stop this kind of error escalating so badly again.

This was unfortunate indeed :-(

Mario and I tested this is two customers but both use IPK. We didn't
consider testing with RPM.

To fix the issue, can we send a depends for native/nativesdk?
Mario Domenech Goulart - Aug. 21, 2014, 12:11 p.m.
On Thu, 21 Aug 2014 08:50:51 +0100 Richard Purdie <richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2014-08-14 at 17:20 -0300, Mario Domenech Goulart wrote:
>> Without this change, meta-openembedded's php recipe (as of 45e62fb8 --
>> "php 5.4.14: use pkg-config for libxml2 detection") breaks with this
>> error on my system:
>> 
>>    tmp/sysroots/x86_64-linux/usr/lib/libxml2.so: undefined reference
>>    to `gzopen64@ZLIB_1.2.3.3'
>> 
>> It seems that the host's zlib is leaking into libxml2's build.
>
> It turns out we need a zlib dependency here, else createrepo fails to
> deal with the compressed index files in the rpm repos. With rpm enabled,
> you can reproduce with:
>
> "bitbake package-index"
>
> Unfortunately this error causes a cascade of other problems and caused
> several people hours of work trying to figure out why the autobuilder
> was "exploding".
>
> Basically, the error above triggers a binary character to be emitted
> into the logs. The error reporting tool chokes on this since JSON can't
> handle the character causing an exception deep in the event handling
> code.
>
> That exception is badly handled and causes bitbake to enter an infinite
> loop, spewing traceback data to the console. That in turn creates
> gigabytes of log files and the autobuilder falls over.
>
> So if anyone wonders why we haven't had patches merging, this is why.
> We're now quite behind with the queue.
>
> Obviously there are fixes to make in multiple parts of the system to
> stop this kind of error escalating so badly again.

Oh, boy.  I'm really sorry for all the trouble this apparently innocent
patch caused.  Unfortunately we didn't catch this issue here, as we
don't use rpm.

Thanks for all the work you guys have put into handling this issue.

I'll try to come up with a better patch.

Best wishes.
Mario

Patch

diff --git a/meta/recipes-core/libxml/libxml2.inc b/meta/recipes-core/libxml/libxml2.inc
index 9aa640b..2992a76 100644
--- a/meta/recipes-core/libxml/libxml2.inc
+++ b/meta/recipes-core/libxml/libxml2.inc
@@ -39,8 +39,8 @@  do_configure_prepend_class-nativesdk () {
 }
 
 EXTRA_OECONF = "--without-python --without-debug --without-legacy --with-catalog --without-docbook --with-c14n --without-lzma --with-fexceptions"
-EXTRA_OECONF_class-native = "--with-python=${STAGING_BINDIR}/python --without-legacy --without-docbook --with-c14n --without-lzma"
-EXTRA_OECONF_class-nativesdk = "--with-python=${STAGING_BINDIR}/python --without-legacy --without-docbook --with-c14n --without-lzma"
+EXTRA_OECONF_class-native = "--with-python=${STAGING_BINDIR}/python --without-legacy --without-docbook --with-c14n --without-lzma --without-zlib"
+EXTRA_OECONF_class-nativesdk = "--with-python=${STAGING_BINDIR}/python --without-legacy --without-docbook --with-c14n --without-lzma --without-zlib"
 EXTRA_OECONF_linuxstdbase = "--without-python --with-debug --with-legacy --with-docbook --with-c14n --without-lzma"
 
 # required for pythong binding