Patchwork util-linux: Remove static libraries from -dev packages

login
register
mail settings
Submitter Phil Blundell
Date Sept. 25, 2012, 12:54 p.m.
Message ID <1348577690.31293.70.camel@phil-desktop>
Download mbox | patch
Permalink /patch/37185/
State New
Headers show

Comments

Phil Blundell - Sept. 25, 2012, 12:54 p.m.
Fixes the QA warnings:

WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'

Signed-off-by: Phil Blundell <pb@pbcl.net>
---
 meta/recipes-core/util-linux/util-linux.inc       |    4 ++--
 meta/recipes-core/util-linux/util-linux_2.21.2.bb |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
Colin Walters - Sept. 25, 2012, 9:59 p.m.
On Tue, 2012-09-25 at 13:54 +0100, Phil Blundell wrote:
> Fixes the QA warnings:
> 
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'

> -FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> +FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"

Wouldn't EXTRA_OECONF += " --disable-static" be better?
Phil Blundell - Sept. 25, 2012, 10 p.m.
On Tue, 2012-09-25 at 17:59 -0400, Colin Walters wrote:
> On Tue, 2012-09-25 at 13:54 +0100, Phil Blundell wrote:
> > Fixes the QA warnings:
> > 
> > WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
> > WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'
> 
> > -FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> > +FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> 
> Wouldn't EXTRA_OECONF += " --disable-static" be better?

That'd be inconsistent with other packages, since we do generally build
and ship the static libraries.  Having a big switch to turn off static
libraries globally seems like a fine plan, but I can't see any obvious
reason why the util-linux ones are any more useless than the rest.

p.
Colin Walters - Sept. 25, 2012, 10:11 p.m.
On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:

> That'd be inconsistent with other packages, since we do generally build
> and ship the static libraries.  Having a big switch to turn off static
> libraries globally seems like a fine plan, but I can't see any obvious
> reason why the util-linux ones are any more useless than the rest.

Makes sense.  I wonder if there are actually any users of the static
libraries.

For what it's worth in gnome-ostree I do just globally pass
--disable-static by default.
Richard Purdie - Sept. 26, 2012, 8:49 a.m.
On Tue, 2012-09-25 at 18:11 -0400, Colin Walters wrote:
> On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
> 
> > That'd be inconsistent with other packages, since we do generally build
> > and ship the static libraries.  Having a big switch to turn off static
> > libraries globally seems like a fine plan, but I can't see any obvious
> > reason why the util-linux ones are any more useless than the rest.
> 
> Makes sense.  I wonder if there are actually any users of the static
> libraries.
> 
> For what it's worth in gnome-ostree I do just globally pass
> --disable-static by default.

I tested this a while back to see what performance difference it made.
The answer was "nothing too significant", I don't have the exact timings
handy. I do remember having to exclude sqlite-native from the list since
pseudo static links against it.

Cheers,

Richard
Phil Blundell - Sept. 26, 2012, 9:28 a.m.
On Wed, 2012-09-26 at 09:49 +0100, Richard Purdie wrote:
> On Tue, 2012-09-25 at 18:11 -0400, Colin Walters wrote:
> > On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
> > 
> > > That'd be inconsistent with other packages, since we do generally build
> > > and ship the static libraries.  Having a big switch to turn off static
> > > libraries globally seems like a fine plan, but I can't see any obvious
> > > reason why the util-linux ones are any more useless than the rest.
> > 
> > Makes sense.  I wonder if there are actually any users of the static
> > libraries.
> > 
> > For what it's worth in gnome-ostree I do just globally pass
> > --disable-static by default.
> 
> I tested this a while back to see what performance difference it made.
> The answer was "nothing too significant", I don't have the exact timings
> handy. I do remember having to exclude sqlite-native from the list since
> pseudo static links against it.

It's slightly surprising that it doesn't make that much of a difference,
given that building static libraries does essentially double the number
of compilations for library code.  Though, of course, glibc doesn't
support --disable-static nowadays and there might be a few other big
packages that have the same issue.

I guess that if you have enough cores, compilation count becomes
something of a non-issue since it's one of the few things that does
parallelize very well.  It might be interesting to repeat the
measurements of --disable-static on a machine with only a few CPUs and
see whether it makes more of a difference there.

Out of interest, why does pseudo static-link against sqlite anyway?

p.
Richard Purdie - Sept. 26, 2012, 2:05 p.m.
On Wed, 2012-09-26 at 10:28 +0100, Phil Blundell wrote:
> On Wed, 2012-09-26 at 09:49 +0100, Richard Purdie wrote:
> > On Tue, 2012-09-25 at 18:11 -0400, Colin Walters wrote:
> > > On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
> > > 
> > > > That'd be inconsistent with other packages, since we do generally build
> > > > and ship the static libraries.  Having a big switch to turn off static
> > > > libraries globally seems like a fine plan, but I can't see any obvious
> > > > reason why the util-linux ones are any more useless than the rest.
> > > 
> > > Makes sense.  I wonder if there are actually any users of the static
> > > libraries.
> > > 
> > > For what it's worth in gnome-ostree I do just globally pass
> > > --disable-static by default.
> > 
> > I tested this a while back to see what performance difference it made.
> > The answer was "nothing too significant", I don't have the exact timings
> > handy. I do remember having to exclude sqlite-native from the list since
> > pseudo static links against it.
> 
> It's slightly surprising that it doesn't make that much of a difference,
> given that building static libraries does essentially double the number
> of compilations for library code.  Though, of course, glibc doesn't
> support --disable-static nowadays and there might be a few other big
> packages that have the same issue.
> 
> I guess that if you have enough cores, compilation count becomes
> something of a non-issue since it's one of the few things that does
> parallelize very well.  It might be interesting to repeat the
> measurements of --disable-static on a machine with only a few CPUs and
> see whether it makes more of a difference there.

Yes, admittedly I probably did test that in a highly parallel
environment and was focusing on the critical path timings as a result.

> Out of interest, why does pseudo static-link against sqlite anyway?

Given the crazy things that pseudo does and the way it works as an ld
preload that keeps itself preloaded, I'm guessing it stops itself having
to worry about finding its own libraries correctly. I just added a
package override for sqlite3-native which wasn't a big deal.

Cheers,

Richard
Mark Hatle - Sept. 26, 2012, 2:50 p.m.
On 9/25/12 5:11 PM, Colin Walters wrote:
> On Tue, 2012-09-25 at 23:00 +0100, Phil Blundell wrote:
>
>> That'd be inconsistent with other packages, since we do generally build
>> and ship the static libraries.  Having a big switch to turn off static
>> libraries globally seems like a fine plan, but I can't see any obvious
>> reason why the util-linux ones are any more useless than the rest.
>
> Makes sense.  I wonder if there are actually any users of the static
> libraries.

Users are primarily small systems, where static linking may result in a slightly 
smaller system.  (When I say small systems, I'm talking about single application 
embedded systems..)

> For what it's worth in gnome-ostree I do just globally pass
> --disable-static by default.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Saul Wold - Sept. 27, 2012, 3:42 p.m.
On 09/25/2012 05:54 AM, Phil Blundell wrote:
> Fixes the QA warnings:
>
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libblkid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libblkid-dev/lib/libblkid.a'
> WARNING: QA Issue: non -staticdev package contains static .a library: util-linux-libuuid-dev path '/work/mips32el-oe-linux/util-linux/2.21.2-r3micro3/packages-split/util-linux-libuuid-dev/lib/libuuid.a'
>
> Signed-off-by: Phil Blundell <pb@pbcl.net>
> ---
>   meta/recipes-core/util-linux/util-linux.inc       |    4 ++--
>   meta/recipes-core/util-linux/util-linux_2.21.2.bb |    2 +-
>   2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/util-linux/util-linux.inc b/meta/recipes-core/util-linux/util-linux.inc
> index 2c5267f..b2a1552 100644
> --- a/meta/recipes-core/util-linux/util-linux.inc
> +++ b/meta/recipes-core/util-linux/util-linux.inc
> @@ -57,9 +57,9 @@ FILES_util-linux-readprofile = "${base_sbindir}/readprofile.${BPN}"
>   FILES_util-linux-uuidgen = "${bindir}/uuidgen"
>
>   FILES_util-linux-libblkid = "${base_libdir}/libblkid.so.*"
> -FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
> +FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
>   FILES_util-linux-libuuid = "${base_libdir}/libuuid.so.*"
> -FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.a ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
> +FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
>   FILES_util-linux-lscpu = "${bindir}/lscpu"
>
>   FILES_util-linux-fsck = "${base_sbindir}/fsck*"
> diff --git a/meta/recipes-core/util-linux/util-linux_2.21.2.bb b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
> index e8f00fc..792730c 100644
> --- a/meta/recipes-core/util-linux/util-linux_2.21.2.bb
> +++ b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
> @@ -1,5 +1,5 @@
>   MAJOR_VERSION = "2.21"
> -PR = "r3"
> +PR = "r4"
>   require util-linux.inc
>
>   # note that `lscpu' is under GPLv3+
>

Merged into OE-Core

Thanks
	Sau!
Phil Blundell - Sept. 30, 2012, 4:22 p.m.
On Wed, 2012-09-26 at 15:05 +0100, Richard Purdie wrote:
> Yes, admittedly I probably did test that in a highly parallel
> environment and was focusing on the critical path timings as a result.

Since one of the machines at the office had nothing better to do on a
Sunday I thought I'd rerun a test with and without static libs on a
smaller system than yours.  I used:

disable_static = "--disable-static"
disable_static_pn-sqlite3-native = ""
disable_static_pn-openssl = ""
disable_static_pn-openssl-native = ""
disable_static_pn-qemu-native = ""
EXTRA_OECONF_append = " ${disable_static}"
BB_NUMBER_THREADS = "4"
PARALLEL_MAKE = "-j4"

and I had to hack the openjade recipe to remove "-a" from a couple of
oe_libinstalls, but apart from that I used an unmodified tree with the
default local.conf from oe-init-build-env.

The timings for "bitbake core-minimal-native" with and without static
libraries were:

13988.74user 2043.25system 1:48:41elapsed 245%CPU (0avgtext+0avgdata
1789392maxresident)k
vs
12876.66user 1931.70system 1:42:43elapsed 240%CPU (0avgtext+0avgdata
1800288maxresident)k

which equates to about an 8% saving in user cycles and about 5.5% in
wall time.  So, not vast, but not totally insignificant either.  You
also get about a 5% saving in TMPDIR size (equivalent to ~ 850MB in this
test).

Obviously, there are some packages which don't use autotools (or
deliberately disregard --disable-static) and hence didn't get their
static libraries disabled by this simplistic configuration.
Specifically, I still have:

tmp-eglibc/deploy/ipk/i586/bison-staticdev_2.5.1-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/e2fsprogs-staticdev_1.42.1-r3_i586.ipk
tmp-eglibc/deploy/ipk/i586/eglibc-staticdev_2.16-r14+svnr20393_i586.ipk
tmp-eglibc/deploy/ipk/i586/elfutils-staticdev_0.148-r9_i586.ipk
tmp-eglibc/deploy/ipk/i586/flex-staticdev_2.5.35-r3_i586.ipk
tmp-eglibc/deploy/ipk/i586/libacl-staticdev_2.2.51-r3_i586.ipk
tmp-eglibc/deploy/ipk/i586/libattr-staticdev_2.4.46-r4_i586.ipk
tmp-eglibc/deploy/ipk/i586/libcap-staticdev_2.22-r5_i586.ipk
tmp-eglibc/deploy/ipk/i586/libstdc++-staticdev_4.7.2-r13_i586.ipk
tmp-eglibc/deploy/ipk/i586/libwrap-staticdev_7.6-r9_i586.ipk
tmp-eglibc/deploy/ipk/i586/libz-staticdev_1.2.7-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/lighttpd-module-staticfile_1.4.31-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/ncurses-staticdev_5.9-r10.1_i586.ipk
tmp-eglibc/deploy/ipk/i586/ocf-linux-staticdev_20100325-r3.0_i586.ipk
tmp-eglibc/deploy/ipk/i586/openssl-staticdev_1.0.0j-r15.3_i586.ipk
tmp-eglibc/deploy/ipk/i586/psmisc-staticdev_22.19-r0_i586.ipk
tmp-eglibc/deploy/ipk/i586/python-distutils-staticdev_2.7.3-r0.2_i586.ipk

p.

Patch

diff --git a/meta/recipes-core/util-linux/util-linux.inc b/meta/recipes-core/util-linux/util-linux.inc
index 2c5267f..b2a1552 100644
--- a/meta/recipes-core/util-linux/util-linux.inc
+++ b/meta/recipes-core/util-linux/util-linux.inc
@@ -57,9 +57,9 @@  FILES_util-linux-readprofile = "${base_sbindir}/readprofile.${BPN}"
 FILES_util-linux-uuidgen = "${bindir}/uuidgen"
 
 FILES_util-linux-libblkid = "${base_libdir}/libblkid.so.*"
-FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.a ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
+FILES_util-linux-libblkid-dev = "${base_libdir}/libblkid.so ${base_libdir}/libblkid.la ${includedir}/blkid ${libdir}/pkgconfig/blkid.pc"
 FILES_util-linux-libuuid = "${base_libdir}/libuuid.so.*"
-FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.a ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
+FILES_util-linux-libuuid-dev = "${base_libdir}/libuuid.so ${base_libdir}/libuuid.la ${includedir}/uuid ${libdir}/pkgconfig/uuid.pc"
 FILES_util-linux-lscpu = "${bindir}/lscpu"
 
 FILES_util-linux-fsck = "${base_sbindir}/fsck*"
diff --git a/meta/recipes-core/util-linux/util-linux_2.21.2.bb b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
index e8f00fc..792730c 100644
--- a/meta/recipes-core/util-linux/util-linux_2.21.2.bb
+++ b/meta/recipes-core/util-linux/util-linux_2.21.2.bb
@@ -1,5 +1,5 @@ 
 MAJOR_VERSION = "2.21"
-PR = "r3"
+PR = "r4"
 require util-linux.inc
 
 # note that `lscpu' is under GPLv3+