Patchwork [1/7] bitbake.conf: fix definition of ${libexecdir}

login
register
mail settings
Submitter Enrico Scholz
Date April 29, 2013, 3:26 p.m.
Message ID <1367249183-2579-1-git-send-email-enrico.scholz@sigma-chemnitz.de>
Download mbox | patch
Permalink /patch/49073/
State New
Headers show

Comments

Enrico Scholz - April 29, 2013, 3:26 p.m.
The GNU standard requires that

  The definition of ‘libexecdir’ is the same for all packages
  [http://www.gnu.org/prep/standards/html_node/Directory-Variables.html]

This means that it must not contain neither:

* multilib qualifiers, nor
* the package name

The recent definition of libexecdir violates this because it can
expand e.g. to

* /usr/lib64/openssh for a 64 bit openssh.bb and
* /usr/lib/dropbear for a 32 bit dropbear.bb

Calls of the 32 bit dropbear to ${libexecdir}/sftp-server will fail
hence.

Patch sets libexecdir to '/usr/lib' as done e.g. by Debian.  Because the
patch changes some basic variables, requires changes of recipes
packaging wildcard '${libexecdir}/*' and can cause silent breakage,
further testing of the generated images is needed and the patch is not
applicable to Dylan.

Wildcard '${libexecdir}/*' in FILES list must be removed after this
change because '${libexecdir}' can become equal to '${libdir}' and these
directories require a different packaging.

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
---
 meta/conf/bitbake.conf | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Ross Burton - April 29, 2013, 3:33 p.m.
Hi,

I agree with dropping $libdir and hard-coding "lib", but have you done
a world oe-core build with buildhistory to verify that nothing else
moved around?

Ross
Enrico Scholz - April 29, 2013, 3:52 p.m.
"Burton, Ross" <ross.burton-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
writes:

> I agree with dropping $libdir and hard-coding "lib", but have you done
> a world oe-core build

no; I do not have the resources for this :( I did a static analysis of
recent recipes and fixed the obvious things.

Change will cause probably some silent breakage which can not be detected
by static analysis.  E.g. the last /usr/libexec -> /usr/lib/${BPN} change
broke dropbear sftp which remained undetected for more than a half year.

We are at the beginning of a new release cycle which is a good time for
this change.


> with buildhistory to verify that nothing else moved around?

Things *are* expected to move around.  E.g. /usr/lib/openssh/sftp-server
becomes /usr/lib/sftp-server now.



Enrico
Ross Burton - April 29, 2013, 4:20 p.m.
Hi,.

On 29 April 2013 16:52, Enrico Scholz <enrico.scholz@sigma-chemnitz.de> wrote:
>> with buildhistory to verify that nothing else moved around?
>
> Things *are* expected to move around.  E.g. /usr/lib/openssh/sftp-server
> becomes /usr/lib/sftp-server now.

Right, so don't use Debian as a rationale for this approach.  On Debian:

$ ls /usr/lib/openssh/
sftp-server  ssh-keysign  ssh-pkcs11-helper

For reference, on Fedora this binary is
/usr/libexec/openssh/sftp-server, with the split between dropping
binaries directly into /usr/libexec or using a subdirectory apparently
arbitrary.

Every package that I looked at on Debian is using
/usr/lib/[packagenameish]/[binary].  /usr/lib/upower/upowerd.
/usr/lib/telepathy/telepathy-gabble.   I say packagenameish as it's
often changed to be a more general name - i.e. all of the Telepathy
backends use /usr/lib/telepathy.

There are no *executables*, only libraries, in /usr/lib on Debian or Fedora.

I'm really not liking dropping the executables into a directory that
on non-multilib is the same as libdir.  If we want to be like the half
of the world that does the same as Fedora then we can revert back to
/usr/libexec.  Or we can be like the half of the world that does the
same as Debian and use something similar to
${prefix}/lib/$(DEB_SOURCE_PACKAGE) (as used in CDBS and Debhelper):

+export libexecdir = "${exec_prefix}/lib/${BPN}"

Ross
Enrico Scholz - April 29, 2013, 5:03 p.m.
"Burton, Ross" <ross.burton-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
writes:

>> Things *are* expected to move around.  E.g. /usr/lib/openssh/sftp-server
>> becomes /usr/lib/sftp-server now.
>
> Right, so don't use Debian as a rationale for this approach.

Debian defines libexecdir to be /usr/lib (+multiarch).  It might set
sometimes per-package '--libexecdir=${libexecdir}/<package>' configure
options, but libexecdir is always /usr/lib there.


> $ ls /usr/lib/openssh/
> sftp-server  ssh-keysign  ssh-pkcs11-helper
>
> For reference, on Fedora this binary is /usr/libexec/openssh/sftp-server,
> with the split between dropping binaries directly into /usr/libexec or
> using a subdirectory apparently arbitrary.

Placing openssh helpers in own subdirectory can make sense when
there are alternative implementations.  Nevertheless, programs
expect ${libexecdir}/sftp-server there.

Every recipe using libexecdir must be investigated whether it is placing
only private files into it (--> candidate for ${pkglibexecdir}), or
whether files are part of its public api (--> must be directly below
${libexecdir}).


> Every package that I looked at on Debian is using
> /usr/lib/[packagenameish]/[binary].  /usr/lib/upower/upowerd.
> /usr/lib/telepathy/telepathy-gabble.  I say packagenameish as it's
> often changed to be a more general name - i.e. all of the Telepathy
> backends use /usr/lib/telepathy.
>
> There are no *executables*, only libraries, in /usr/lib on Debian or

I do not have Debian for reference; but Ubuntu-12.04 (only a minimal,
GUI-less installation) has

/usr/lib/command-not-found
/usr/lib/pt_chown

and

/usr/lib/sftp-server -> openssh/sftp-server



> Fedora.

Fedora has /usr/libexec with lot of executables directly below it.


> I'm really not liking dropping the executables into a directory that
> on non-multilib is the same as libdir.

When we want to follow FHS strictly, nothing prevents us from placing
files into /usr/lib.

  /usr/lib includes object files, libraries, and internal binaries
  that are not intended to be executed directly by users or shell
  scripts.

  Applications may use a single subdirectory under /usr/lib. If an
  application uses a subdirectory, all architecture-dependent data
  exclusively used by the application must be placed within that
  subdirectory.


When we really do not want executables under ${libdir}, we can set

  libexecdir = ${prefix}/lib/libexec

(which might be a good setting for world-builds to detect some kind of
libexecdir related problems).


> If we want to be like the half of the world that does the same as
> Fedora then we can revert back to /usr/libexec.  Or we can be like
> the half of the world that does the same as Debian and use something
> similar to ${prefix}/lib/$(DEB_SOURCE_PACKAGE) (as used in CDBS and
> Debhelper):

Debian does *not* include the package name since mid 2011

        http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541458


Enrico
Saul Wold - April 29, 2013, 10:33 p.m.
On 04/29/2013 10:03 AM, Enrico Scholz wrote:
> "Burton, Ross" <ross.burton-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> writes:
>
>>> Things *are* expected to move around.  E.g. /usr/lib/openssh/sftp-server
>>> becomes /usr/lib/sftp-server now.
>>
>> Right, so don't use Debian as a rationale for this approach.
>
> Debian defines libexecdir to be /usr/lib (+multiarch).  It might set
> sometimes per-package '--libexecdir=${libexecdir}/<package>' configure
> options, but libexecdir is always /usr/lib there.
>

As I read though this, a couple things I noted from some of the 
available information, the GNU Directory section you quoted earlier 
actually suggests a package-name directory should be included.

   The definition of ‘libexecdir’ is the same for all packages, so you
   should install your data in a subdirectory thereof. Most packages
   install their data under $(libexecdir)/package-name/, possibly within
   additional subdirectories thereof, such as $(libexecdir)/package-
   name/machine/version.

And as you pointed out below, the FHS also touches on having a 
sub-directory for applications to use for non-user object and internal 
binaries (like plugins).


>
>> $ ls /usr/lib/openssh/
>> sftp-server  ssh-keysign  ssh-pkcs11-helper
>>
>> For reference, on Fedora this binary is /usr/libexec/openssh/sftp-server,
>> with the split between dropping binaries directly into /usr/libexec or
>> using a subdirectory apparently arbitrary.
>
> Placing openssh helpers in own subdirectory can make sense when
> there are alternative implementations.  Nevertheless, programs
> expect ${libexecdir}/sftp-server there.
>
There seems to be a specific case here with openssh and dropbear having 
alternatives for similar binaries provided by multiple packages, then we 
should handle that special case.

> Every recipe using libexecdir must be investigated whether it is placing
> only private files into it (--> candidate for ${pkglibexecdir}), or
> whether files are part of its public api (--> must be directly below
> ${libexecdir}).
>
>
>> Every package that I looked at on Debian is using
>> /usr/lib/[packagenameish]/[binary].  /usr/lib/upower/upowerd.
>> /usr/lib/telepathy/telepathy-gabble.  I say packagenameish as it's
>> often changed to be a more general name - i.e. all of the Telepathy
>> backends use /usr/lib/telepathy.
>>
>> There are no *executables*, only libraries, in /usr/lib on Debian or
>
> I do not have Debian for reference; but Ubuntu-12.04 (only a minimal,
> GUI-less installation) has
>
> /usr/lib/command-not-found
> /usr/lib/pt_chown
>
> and
>
> /usr/lib/sftp-server -> openssh/sftp-server
>
This could be done via the update-alternatives mechanism in the oe-core 
space.

>
>
>> Fedora.
>
> Fedora has /usr/libexec with lot of executables directly below it.
>
>
>> I'm really not liking dropping the executables into a directory that
>> on non-multilib is the same as libdir.
>
> When we want to follow FHS strictly, nothing prevents us from placing
> files into /usr/lib.
>
>    /usr/lib includes object files, libraries, and internal binaries
>    that are not intended to be executed directly by users or shell
>    scripts.
>
>    Applications may use a single subdirectory under /usr/lib. If an
>    application uses a subdirectory, all architecture-dependent data
>    exclusively used by the application must be placed within that
>    subdirectory.
>
>
> When we really do not want executables under ${libdir}, we can set
>
>    libexecdir = ${prefix}/lib/libexec
>
> (which might be a good setting for world-builds to detect some kind of
> libexecdir related problems).
>
I prefer what Ross suggested, fixing libexecdir to be 
${exec_prefix}/lib/${BPN}

>
>> If we want to be like the half of the world that does the same as
>> Fedora then we can revert back to /usr/libexec.  Or we can be like
>> the half of the world that does the same as Debian and use something
>> similar to ${prefix}/lib/$(DEB_SOURCE_PACKAGE) (as used in CDBS and
>> Debhelper):
>
> Debian does *not* include the package name since mid 2011
>
>          http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541458
>
I just looked at a Debian build and /usr/lib is abound with package 
directories. So given the direction we already have going with the half 
that does ${prefix}/lib/${package name} makes the most sense.  We are 
not exactly Fedora nor Debian, OE-Core is an embedded system, we are 
doing our best to match with the FHS as LSB is one of the design features.

Since I did the last libexecdir change, I will start to investigate 
changing libexecdir to be ${exec_prefix}/lib/${BPN}.  The concern I have 
with this will be plugins and other executable binaries created in a 
multilib environment.

Sau!

>
> Enrico
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
Enrico Scholz - April 29, 2013, 11:15 p.m.
Saul Wold <sgw@linux.intel.com> writes:

>>>> Things *are* expected to move around.  E.g. /usr/lib/openssh/sftp-server
>>>> becomes /usr/lib/sftp-server now.
>>>
>>> Right, so don't use Debian as a rationale for this approach.
>>
>> Debian defines libexecdir to be /usr/lib (+multiarch).  It might set
>> sometimes per-package '--libexecdir=${libexecdir}/<package>' configure
>> options, but libexecdir is always /usr/lib there.
>
> As I read though this, a couple things I noted from some of the
> available information, the GNU Directory section you quoted earlier
> actually suggests a package-name directory should be included.

Yes; automake calls this

  pkglibexecdir = $(libexecdir)/@PACKAGE@

and lot of packages are using it already to install their files.


>   The definition of ‘libexecdir’ is the same for all packages, so you
>   should install your data in a subdirectory thereof. Most packages
>   install their data under $(libexecdir)/package-name/, possibly within
>   additional subdirectories thereof, such as $(libexecdir)/package-
>   name/machine/version.
>
> And as you pointed out below, the FHS also touches on having a
> sub-directory for applications to use for non-user object and
> internal binaries (like plugins).

Yes; but this is thing of the package, not of the distribution.  When a
distribution thinks, a package pollutes $(libexecdir), the distribution
can fix *this* package.  But $(libexecdir) provided by the distribution
must never contain the package name.


>> When we really do not want executables under ${libdir}, we can set
>>
>>    libexecdir = ${prefix}/lib/libexec
>>
>> (which might be a good setting for world-builds to detect some kind of
>> libexecdir related problems).
>>
> I prefer what Ross suggested, fixing libexecdir to be
> ${exec_prefix}/lib/${BPN}

$(libexecdir) has the same meaning like $(bindir); so why do we not have

  bindir = $(prefix)/bin/${BPN}

?

Again: $(libexecdir) must be the same for every package.  All major
linux distributions follow this; why should oe differ from this rule?


>>> If we want to be like the half of the world that does the same as
>>> Fedora then we can revert back to /usr/libexec.  Or we can be like
>>> the half of the world that does the same as Debian and use something
>>> similar to ${prefix}/lib/$(DEB_SOURCE_PACKAGE) (as used in CDBS and
>>> Debhelper):
>>
>> Debian does *not* include the package name since mid 2011
>>
>>          http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541458
>>
> I just looked at a Debian build and /usr/lib is abound with package
> directories.

Nobody forbids this and it is completely fine and suggested that
packages install their files below $(pkglibexecdir).  But Debian
defines libexecdir as /usr/lib (plus an optional multiarch tag from
DEB_HOST_MULTIARCH (whatever this is)).


> Since I did the last libexecdir change, I will start to investigate
> changing libexecdir to be ${exec_prefix}/lib/${BPN}.

This is a no-go because it breaks the GNU standard like the recent
value.  fwiw, my preferences for libexecdir are (in this order):

1. ${prefix}/libexec

   --> clean separation, commonly used, but not FHS compliant (unless
   somebody interprets "/usr/lib<qual> : Alternate format libraries
   (optional)" with '<qual>=exec')

2. ${prefix}/lib

   --> follows the Debian way and is 100% FHS compliant, but might
   pollute ${libdir} and might introduce regressions in oe packaging

3. ${prefix}/lib/libexec

   --> clean separation, probably FHS compliant, but smells hacky


> The concern I have with this will be plugins and other executable
> binaries created in a multilib environment.

Plugins (aka "libraries to be loaded") are not interesting (they do not
belong into ${libexecdir}).  Executables within ${libexecdir} are to be
handled like these in ${bindir}, so that a proper libexecdir won't cause
any new multilib issue.



Enrico
Phil Blundell - April 30, 2013, 9:34 a.m.
On Tue, 2013-04-30 at 01:15 +0200, Enrico Scholz wrote:
> 1. ${prefix}/libexec
> 
>    --> clean separation, commonly used, but not FHS compliant (unless
>    somebody interprets "/usr/lib<qual> : Alternate format libraries
>    (optional)" with '<qual>=exec')

Actually, FHS 3.0 draft 1 does appear to admit the possibility of
libexec again:

http://www.linuxbase.org/betaspecs/fhs/fhs.html#usrlibexec

So maybe this option isn't (or won't be) so noncompliant as it might
first have appeared.

p.

Patch

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1f4fbc5..9b864ec 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -38,7 +38,7 @@  export systemd_unitdir = "/lib/systemd"
 export bindir = "${exec_prefix}/bin"
 export sbindir = "${exec_prefix}/sbin"
 export libdir = "${exec_prefix}/${baselib}"
-export libexecdir = "${libdir}/${BPN}"
+export libexecdir = "${exec_prefix}/lib"
 export includedir = "${exec_prefix}/include"
 export oldincludedir = "${exec_prefix}/include"
 localedir = "${libdir}/locale"
@@ -52,7 +52,7 @@  bindir_native = "${prefix_native}/bin"
 sbindir_native = "${prefix_native}/sbin"
 includedir_native = "${prefix_native}/include"
 libdir_native = "${prefix_native}/lib"
-libexecdir_native = "${libdir_native}/${BPN}"
+libexecdir_native = "${prefix_native}/lib"
 base_libdir_native = "/lib"
 datadir_native = "${prefix_native}/share"
 bindir_cross = "/bin"