[v1] postinst-intercepts: check tool presence in intercept hooks

Submitted by Sinan Kaya on June 25, 2019, 10:13 p.m. | Patch ID: 162518

Details

Message ID 20190625221319.102643-1-okaya@kernel.org
State New
Headers show

Commit Message

Sinan Kaya June 25, 2019, 10:13 p.m.
do_rootfs: The postinstall intercept hook 'update_udev_hwdb'
do_rootfs: The postinstall intercept hook 'update_gio_module_cache' failed
do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
do_rootfs: The postinstall intercept hook 'update_icon_cache' failed

Signed-off-by: Sinan Kaya <okaya@kernel.org>
---
 scripts/postinst-intercepts/update_font_cache    |  6 ++++--
 .../postinst-intercepts/update_gio_module_cache  | 10 +++++++---
 scripts/postinst-intercepts/update_icon_cache    | 16 +++++++++-------
 scripts/postinst-intercepts/update_pixbuf_cache  |  5 ++++-
 scripts/postinst-intercepts/update_udev_hwdb     |  6 ++++--
 5 files changed, 28 insertions(+), 15 deletions(-)

Patch hide | download patch | download mbox

diff --git a/scripts/postinst-intercepts/update_font_cache b/scripts/postinst-intercepts/update_font_cache
index 3053c7065f4..954acbb41b2 100644
--- a/scripts/postinst-intercepts/update_font_cache
+++ b/scripts/postinst-intercepts/update_font_cache
@@ -5,5 +5,7 @@ 
 
 set -e
 
-PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache --sysroot=$D --system-only ${fontconfigcacheparams}
-chown -R root:root $D${fontconfigcachedir}
+if [ -f $D${libexecdir}/${binprefix}fc-cache ]; then
+	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache --sysroot=$D --system-only ${fontconfigcacheparams}
+	chown -R root:root $D${fontconfigcachedir}
+fi
diff --git a/scripts/postinst-intercepts/update_gio_module_cache b/scripts/postinst-intercepts/update_gio_module_cache
index c87fa85db9e..e8483cb0bf5 100644
--- a/scripts/postinst-intercepts/update_gio_module_cache
+++ b/scripts/postinst-intercepts/update_gio_module_cache
@@ -5,7 +5,11 @@ 
 
 set -e
 
-PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
+if [ -f $D${libexecdir}/${binprefix}gio-querymodules ]; then
+
+	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
+
+	[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
+		chown root:root $D${libdir}/gio/modules/giomodule.cache
+fi
 
-[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
-	chown root:root $D${libdir}/gio/modules/giomodule.cache
diff --git a/scripts/postinst-intercepts/update_icon_cache b/scripts/postinst-intercepts/update_icon_cache
index 212209a9daf..a67ff7ef970 100644
--- a/scripts/postinst-intercepts/update_icon_cache
+++ b/scripts/postinst-intercepts/update_icon_cache
@@ -5,12 +5,14 @@ 
 
 set -e
 
-# update native pixbuf loaders
-$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders --update-cache
+if [ -d $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0 ]; then
+	# update native pixbuf loaders
+	$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders --update-cache
 
-for icondir in $D/usr/share/icons/*/ ; do
-    if [ -d $icondir ] ; then
-        gtk-update-icon-cache -fqt  $icondir
-    fi
-done
+	for icondir in $D/usr/share/icons/*/ ; do
+	    if [ -d $icondir ] ; then
+	        gtk-update-icon-cache -fqt  $icondir
+	    fi
+	done
+fi
 
diff --git a/scripts/postinst-intercepts/update_pixbuf_cache b/scripts/postinst-intercepts/update_pixbuf_cache
index ea12814474e..166422e5c37 100644
--- a/scripts/postinst-intercepts/update_pixbuf_cache
+++ b/scripts/postinst-intercepts/update_pixbuf_cache
@@ -8,6 +8,9 @@  set -e
 export GDK_PIXBUF_MODULEDIR=$D${libdir}/gdk-pixbuf-2.0/2.10.0/loaders
 export GDK_PIXBUF_FATAL_LOADER=1
 
-PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
+if [ -d $D${libdir}/gdk-pixbuf-2.0 ]; then
+	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
     >$GDK_PIXBUF_MODULEDIR/../loaders.cache && \
     sed -i -e "s:$D::g" $GDK_PIXBUF_MODULEDIR/../loaders.cache
+fi
+
diff --git a/scripts/postinst-intercepts/update_udev_hwdb b/scripts/postinst-intercepts/update_udev_hwdb
index c4fb2bffcbf..7b6e38338f3 100644
--- a/scripts/postinst-intercepts/update_udev_hwdb
+++ b/scripts/postinst-intercepts/update_udev_hwdb
@@ -5,5 +5,7 @@ 
 
 set -e
 
-PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
-chown root:root $D${sysconfdir}/udev/hwdb.bin
+if [ -d $D/lib/udev/hwdb.d ]; then
+	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
+	chown root:root $D${sysconfdir}/udev/hwdb.bin
+fi

Comments

Qi.Chen@windriver.com June 26, 2019, 2:27 a.m.
Under which circumstances will the tools be absent?
And why do you check the existence of files for some the the existence 
of directories for others?

Best Regards,
Chen Qi

On 06/26/2019 06:13 AM, Sinan Kaya wrote:
> do_rootfs: The postinstall intercept hook 'update_udev_hwdb'
> do_rootfs: The postinstall intercept hook 'update_gio_module_cache' failed
> do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
> do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
>
> Signed-off-by: Sinan Kaya <okaya@kernel.org>
> ---
>   scripts/postinst-intercepts/update_font_cache    |  6 ++++--
>   .../postinst-intercepts/update_gio_module_cache  | 10 +++++++---
>   scripts/postinst-intercepts/update_icon_cache    | 16 +++++++++-------
>   scripts/postinst-intercepts/update_pixbuf_cache  |  5 ++++-
>   scripts/postinst-intercepts/update_udev_hwdb     |  6 ++++--
>   5 files changed, 28 insertions(+), 15 deletions(-)
>
> diff --git a/scripts/postinst-intercepts/update_font_cache b/scripts/postinst-intercepts/update_font_cache
> index 3053c7065f4..954acbb41b2 100644
> --- a/scripts/postinst-intercepts/update_font_cache
> +++ b/scripts/postinst-intercepts/update_font_cache
> @@ -5,5 +5,7 @@
>   
>   set -e
>   
> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache --sysroot=$D --system-only ${fontconfigcacheparams}
> -chown -R root:root $D${fontconfigcachedir}
> +if [ -f $D${libexecdir}/${binprefix}fc-cache ]; then
> +	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache --sysroot=$D --system-only ${fontconfigcacheparams}
> +	chown -R root:root $D${fontconfigcachedir}
> +fi
> diff --git a/scripts/postinst-intercepts/update_gio_module_cache b/scripts/postinst-intercepts/update_gio_module_cache
> index c87fa85db9e..e8483cb0bf5 100644
> --- a/scripts/postinst-intercepts/update_gio_module_cache
> +++ b/scripts/postinst-intercepts/update_gio_module_cache
> @@ -5,7 +5,11 @@
>   
>   set -e
>   
> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
> +if [ -f $D${libexecdir}/${binprefix}gio-querymodules ]; then
> +
> +	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
> +
> +	[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
> +		chown root:root $D${libdir}/gio/modules/giomodule.cache
> +fi
>   
> -[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
> -	chown root:root $D${libdir}/gio/modules/giomodule.cache
> diff --git a/scripts/postinst-intercepts/update_icon_cache b/scripts/postinst-intercepts/update_icon_cache
> index 212209a9daf..a67ff7ef970 100644
> --- a/scripts/postinst-intercepts/update_icon_cache
> +++ b/scripts/postinst-intercepts/update_icon_cache
> @@ -5,12 +5,14 @@
>   
>   set -e
>   
> -# update native pixbuf loaders
> -$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders --update-cache
> +if [ -d $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0 ]; then
> +	# update native pixbuf loaders
> +	$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders --update-cache
>   
> -for icondir in $D/usr/share/icons/*/ ; do
> -    if [ -d $icondir ] ; then
> -        gtk-update-icon-cache -fqt  $icondir
> -    fi
> -done
> +	for icondir in $D/usr/share/icons/*/ ; do
> +	    if [ -d $icondir ] ; then
> +	        gtk-update-icon-cache -fqt  $icondir
> +	    fi
> +	done
> +fi
>   
> diff --git a/scripts/postinst-intercepts/update_pixbuf_cache b/scripts/postinst-intercepts/update_pixbuf_cache
> index ea12814474e..166422e5c37 100644
> --- a/scripts/postinst-intercepts/update_pixbuf_cache
> +++ b/scripts/postinst-intercepts/update_pixbuf_cache
> @@ -8,6 +8,9 @@ set -e
>   export GDK_PIXBUF_MODULEDIR=$D${libdir}/gdk-pixbuf-2.0/2.10.0/loaders
>   export GDK_PIXBUF_FATAL_LOADER=1
>   
> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
> +if [ -d $D${libdir}/gdk-pixbuf-2.0 ]; then
> +	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
>       >$GDK_PIXBUF_MODULEDIR/../loaders.cache && \
>       sed -i -e "s:$D::g" $GDK_PIXBUF_MODULEDIR/../loaders.cache
> +fi
> +
> diff --git a/scripts/postinst-intercepts/update_udev_hwdb b/scripts/postinst-intercepts/update_udev_hwdb
> index c4fb2bffcbf..7b6e38338f3 100644
> --- a/scripts/postinst-intercepts/update_udev_hwdb
> +++ b/scripts/postinst-intercepts/update_udev_hwdb
> @@ -5,5 +5,7 @@
>   
>   set -e
>   
> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
> -chown root:root $D${sysconfdir}/udev/hwdb.bin
> +if [ -d $D/lib/udev/hwdb.d ]; then
> +	PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
> +	chown root:root $D${sysconfdir}/udev/hwdb.bin
> +fi
Alexander Kanavin June 26, 2019, 8:59 a.m.
Yes, this looks suspicious to me as well. If the tools are absent you need
to investigate why they are absent; skipping the tools execution is just
sweeping the problem under the carpet, not fixing it.

Alex

On Wed, 26 Jun 2019 at 04:15, ChenQi <Qi.Chen@windriver.com> wrote:

> Under which circumstances will the tools be absent?
> And why do you check the existence of files for some the the existence
> of directories for others?
>
> Best Regards,
> Chen Qi
>
> On 06/26/2019 06:13 AM, Sinan Kaya wrote:
> > do_rootfs: The postinstall intercept hook 'update_udev_hwdb'
> > do_rootfs: The postinstall intercept hook 'update_gio_module_cache'
> failed
> > do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
> > do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
> >
> > Signed-off-by: Sinan Kaya <okaya@kernel.org>
> > ---
> >   scripts/postinst-intercepts/update_font_cache    |  6 ++++--
> >   .../postinst-intercepts/update_gio_module_cache  | 10 +++++++---
> >   scripts/postinst-intercepts/update_icon_cache    | 16 +++++++++-------
> >   scripts/postinst-intercepts/update_pixbuf_cache  |  5 ++++-
> >   scripts/postinst-intercepts/update_udev_hwdb     |  6 ++++--
> >   5 files changed, 28 insertions(+), 15 deletions(-)
> >
> > diff --git a/scripts/postinst-intercepts/update_font_cache
> b/scripts/postinst-intercepts/update_font_cache
> > index 3053c7065f4..954acbb41b2 100644
> > --- a/scripts/postinst-intercepts/update_font_cache
> > +++ b/scripts/postinst-intercepts/update_font_cache
> > @@ -5,5 +5,7 @@
> >
> >   set -e
> >
> > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E ${fontconfigcacheenv}
> $D${libexecdir}/${binprefix}fc-cache --sysroot=$D --system-only
> ${fontconfigcacheparams}
> > -chown -R root:root $D${fontconfigcachedir}
> > +if [ -f $D${libexecdir}/${binprefix}fc-cache ]; then
> > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E
> ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache --sysroot=$D
> --system-only ${fontconfigcacheparams}
> > +     chown -R root:root $D${fontconfigcachedir}
> > +fi
> > diff --git a/scripts/postinst-intercepts/update_gio_module_cache
> b/scripts/postinst-intercepts/update_gio_module_cache
> > index c87fa85db9e..e8483cb0bf5 100644
> > --- a/scripts/postinst-intercepts/update_gio_module_cache
> > +++ b/scripts/postinst-intercepts/update_gio_module_cache
> > @@ -5,7 +5,11 @@
> >
> >   set -e
> >
> > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
> $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
> > +if [ -f $D${libexecdir}/${binprefix}gio-querymodules ]; then
> > +
> > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
> $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
> > +
> > +     [ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
> > +             chown root:root $D${libdir}/gio/modules/giomodule.cache
> > +fi
> >
> > -[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
> > -     chown root:root $D${libdir}/gio/modules/giomodule.cache
> > diff --git a/scripts/postinst-intercepts/update_icon_cache
> b/scripts/postinst-intercepts/update_icon_cache
> > index 212209a9daf..a67ff7ef970 100644
> > --- a/scripts/postinst-intercepts/update_icon_cache
> > +++ b/scripts/postinst-intercepts/update_icon_cache
> > @@ -5,12 +5,14 @@
> >
> >   set -e
> >
> > -# update native pixbuf loaders
> >
> -$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
> --update-cache
> > +if [ -d $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0 ]; then
> > +     # update native pixbuf loaders
> > +
>  $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
> --update-cache
> >
> > -for icondir in $D/usr/share/icons/*/ ; do
> > -    if [ -d $icondir ] ; then
> > -        gtk-update-icon-cache -fqt  $icondir
> > -    fi
> > -done
> > +     for icondir in $D/usr/share/icons/*/ ; do
> > +         if [ -d $icondir ] ; then
> > +             gtk-update-icon-cache -fqt  $icondir
> > +         fi
> > +     done
> > +fi
> >
> > diff --git a/scripts/postinst-intercepts/update_pixbuf_cache
> b/scripts/postinst-intercepts/update_pixbuf_cache
> > index ea12814474e..166422e5c37 100644
> > --- a/scripts/postinst-intercepts/update_pixbuf_cache
> > +++ b/scripts/postinst-intercepts/update_pixbuf_cache
> > @@ -8,6 +8,9 @@ set -e
> >   export GDK_PIXBUF_MODULEDIR=$D${libdir}/gdk-pixbuf-2.0/2.10.0/loaders
> >   export GDK_PIXBUF_FATAL_LOADER=1
> >
> > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
> $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
> > +if [ -d $D${libdir}/gdk-pixbuf-2.0 ]; then
> > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
> $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
> >       >$GDK_PIXBUF_MODULEDIR/../loaders.cache && \
> >       sed -i -e "s:$D::g" $GDK_PIXBUF_MODULEDIR/../loaders.cache
> > +fi
> > +
> > diff --git a/scripts/postinst-intercepts/update_udev_hwdb
> b/scripts/postinst-intercepts/update_udev_hwdb
> > index c4fb2bffcbf..7b6e38338f3 100644
> > --- a/scripts/postinst-intercepts/update_udev_hwdb
> > +++ b/scripts/postinst-intercepts/update_udev_hwdb
> > @@ -5,5 +5,7 @@
> >
> >   set -e
> >
> > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
> $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
> > -chown root:root $D${sysconfdir}/udev/hwdb.bin
> > +if [ -d $D/lib/udev/hwdb.d ]; then
> > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
> $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
> > +     chown root:root $D${sysconfdir}/udev/hwdb.bin
> > +fi
>
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
Sinan Kaya June 26, 2019, 5:33 p.m.
Here is my investigation:

The intercepts are executed on QEMU during cross-compilation using
qemu-wrapper.

I see that the tools were built for the native architecture but
of course they do not exist on the target image and I have no
interest in pulling fonts and such into my image to make bitbake
happy.

DEBUG: Collected intercepts:
  poky/scripts/postinst-intercepts/update_font_cache
  poky/scripts/postinst-intercepts/update_gio_module_cache
  poky/scripts/postinst-intercepts/update_gtk_immodules_cache
  poky/scripts/postinst-intercepts/update_icon_cache
  poky/scripts/postinst-intercepts/update_pixbuf_cache
  poky/scripts/postinst-intercepts/update_udev_hwdb

I also see that other intercepts are doing the same folder/executable
checks.

What is so special about these?


On 6/26/2019 4:59 AM, Alexander Kanavin wrote:
> Yes, this looks suspicious to me as well. If the tools are absent you
> need to investigate why they are absent; skipping the tools execution is
> just sweeping the problem under the carpet, not fixing it.
> 
> Alex
> 
> On Wed, 26 Jun 2019 at 04:15, ChenQi <Qi.Chen@windriver.com
> <mailto:Qi.Chen@windriver.com>> wrote:
> 
>     Under which circumstances will the tools be absent?
>     And why do you check the existence of files for some the the existence
>     of directories for others?
> 
>     Best Regards,
>     Chen Qi
> 
>     On 06/26/2019 06:13 AM, Sinan Kaya wrote:
>     > do_rootfs: The postinstall intercept hook 'update_udev_hwdb'
>     > do_rootfs: The postinstall intercept hook
>     'update_gio_module_cache' failed
>     > do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
>     > do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
>     >
>     > Signed-off-by: Sinan Kaya <okaya@kernel.org <mailto:okaya@kernel.org>>
>     > ---
>     >   scripts/postinst-intercepts/update_font_cache    |  6 ++++--
>     >   .../postinst-intercepts/update_gio_module_cache  | 10 +++++++---
>     >   scripts/postinst-intercepts/update_icon_cache    | 16
>     +++++++++-------
>     >   scripts/postinst-intercepts/update_pixbuf_cache  |  5 ++++-
>     >   scripts/postinst-intercepts/update_udev_hwdb     |  6 ++++--
>     >   5 files changed, 28 insertions(+), 15 deletions(-)
>     >
>     > diff --git a/scripts/postinst-intercepts/update_font_cache
>     b/scripts/postinst-intercepts/update_font_cache
>     > index 3053c7065f4..954acbb41b2 100644
>     > --- a/scripts/postinst-intercepts/update_font_cache
>     > +++ b/scripts/postinst-intercepts/update_font_cache
>     > @@ -5,5 +5,7 @@
>     >   
>     >   set -e
>     >   
>     > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E
>     ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache
>     --sysroot=$D --system-only ${fontconfigcacheparams}
>     > -chown -R root:root $D${fontconfigcachedir}
>     > +if [ -f $D${libexecdir}/${binprefix}fc-cache ]; then
>     > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E
>     ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache
>     --sysroot=$D --system-only ${fontconfigcacheparams}
>     > +     chown -R root:root $D${fontconfigcachedir}
>     > +fi
>     > diff --git a/scripts/postinst-intercepts/update_gio_module_cache
>     b/scripts/postinst-intercepts/update_gio_module_cache
>     > index c87fa85db9e..e8483cb0bf5 100644
>     > --- a/scripts/postinst-intercepts/update_gio_module_cache
>     > +++ b/scripts/postinst-intercepts/update_gio_module_cache
>     > @@ -5,7 +5,11 @@
>     >   
>     >   set -e
>     >   
>     > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>     $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
>     > +if [ -f $D${libexecdir}/${binprefix}gio-querymodules ]; then
>     > +
>     > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>     $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
>     > +
>     > +     [ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
>     > +             chown root:root $D${libdir}/gio/modules/giomodule.cache
>     > +fi
>     >   
>     > -[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
>     > -     chown root:root $D${libdir}/gio/modules/giomodule.cache
>     > diff --git a/scripts/postinst-intercepts/update_icon_cache
>     b/scripts/postinst-intercepts/update_icon_cache
>     > index 212209a9daf..a67ff7ef970 100644
>     > --- a/scripts/postinst-intercepts/update_icon_cache
>     > +++ b/scripts/postinst-intercepts/update_icon_cache
>     > @@ -5,12 +5,14 @@
>     >   
>     >   set -e
>     >   
>     > -# update native pixbuf loaders
>     >
>     -$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
>     --update-cache
>     > +if [ -d $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0 ]; then
>     > +     # update native pixbuf loaders
>     > +   
>      $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
>     --update-cache
>     >   
>     > -for icondir in $D/usr/share/icons/*/ ; do
>     > -    if [ -d $icondir ] ; then
>     > -        gtk-update-icon-cache -fqt  $icondir
>     > -    fi
>     > -done
>     > +     for icondir in $D/usr/share/icons/*/ ; do
>     > +         if [ -d $icondir ] ; then
>     > +             gtk-update-icon-cache -fqt  $icondir
>     > +         fi
>     > +     done
>     > +fi
>     >   
>     > diff --git a/scripts/postinst-intercepts/update_pixbuf_cache
>     b/scripts/postinst-intercepts/update_pixbuf_cache
>     > index ea12814474e..166422e5c37 100644
>     > --- a/scripts/postinst-intercepts/update_pixbuf_cache
>     > +++ b/scripts/postinst-intercepts/update_pixbuf_cache
>     > @@ -8,6 +8,9 @@ set -e
>     >   export
>     GDK_PIXBUF_MODULEDIR=$D${libdir}/gdk-pixbuf-2.0/2.10.0/loaders
>     >   export GDK_PIXBUF_FATAL_LOADER=1
>     >   
>     > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>     $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
>     > +if [ -d $D${libdir}/gdk-pixbuf-2.0 ]; then
>     > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>     $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
>     >       >$GDK_PIXBUF_MODULEDIR/../loaders.cache && \
>     >       sed -i -e "s:$D::g" $GDK_PIXBUF_MODULEDIR/../loaders.cache
>     > +fi
>     > +
>     > diff --git a/scripts/postinst-intercepts/update_udev_hwdb
>     b/scripts/postinst-intercepts/update_udev_hwdb
>     > index c4fb2bffcbf..7b6e38338f3 100644
>     > --- a/scripts/postinst-intercepts/update_udev_hwdb
>     > +++ b/scripts/postinst-intercepts/update_udev_hwdb
>     > @@ -5,5 +5,7 @@
>     >   
>     >   set -e
>     >   
>     > -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>     $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
>     > -chown root:root $D${sysconfdir}/udev/hwdb.bin
>     > +if [ -d $D/lib/udev/hwdb.d ]; then
>     > +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>     $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
>     > +     chown root:root $D${sysconfdir}/udev/hwdb.bin
>     > +fi
> 
> 
>     -- 
>     _______________________________________________
>     Openembedded-core mailing list
>     Openembedded-core@lists.openembedded.org
>     <mailto:Openembedded-core@lists.openembedded.org>
>     http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
Alexander Kanavin June 26, 2019, 6:10 p.m.
Basically they are executed only when you have something in your target image that requires their execution (usually to populate some database). If the execution fails for some reason, the correct way to handle that is to look into the failure instead of fixing the symptoms (missing files for instance).

Alex

> On 26 Jun 2019, at 19.33, Sinan Kaya <Okaya@kernel.org> wrote:
> 
> Here is my investigation:
> 
> The intercepts are executed on QEMU during cross-compilation using
> qemu-wrapper.
> 
> I see that the tools were built for the native architecture but
> of course they do not exist on the target image and I have no
> interest in pulling fonts and such into my image to make bitbake
> happy.
> 
> DEBUG: Collected intercepts:
>  poky/scripts/postinst-intercepts/update_font_cache
>  poky/scripts/postinst-intercepts/update_gio_module_cache
>  poky/scripts/postinst-intercepts/update_gtk_immodules_cache
>  poky/scripts/postinst-intercepts/update_icon_cache
>  poky/scripts/postinst-intercepts/update_pixbuf_cache
>  poky/scripts/postinst-intercepts/update_udev_hwdb
> 
> I also see that other intercepts are doing the same folder/executable
> checks.
> 
> What is so special about these?
> 
> 
>> On 6/26/2019 4:59 AM, Alexander Kanavin wrote:
>> Yes, this looks suspicious to me as well. If the tools are absent you
>> need to investigate why they are absent; skipping the tools execution is
>> just sweeping the problem under the carpet, not fixing it.
>> 
>> Alex
>> 
>> On Wed, 26 Jun 2019 at 04:15, ChenQi <Qi.Chen@windriver.com
>> <mailto:Qi.Chen@windriver.com>> wrote:
>> 
>>    Under which circumstances will the tools be absent?
>>    And why do you check the existence of files for some the the existence
>>    of directories for others?
>> 
>>    Best Regards,
>>    Chen Qi
>> 
>>>    On 06/26/2019 06:13 AM, Sinan Kaya wrote:
>>> do_rootfs: The postinstall intercept hook 'update_udev_hwdb'
>>> do_rootfs: The postinstall intercept hook
>>    'update_gio_module_cache' failed
>>> do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
>>> do_rootfs: The postinstall intercept hook 'update_icon_cache' failed
>>> 
>>> Signed-off-by: Sinan Kaya <okaya@kernel.org <mailto:okaya@kernel.org>>
>>> ---
>>>    scripts/postinst-intercepts/update_font_cache    |  6 ++++--
>>>    .../postinst-intercepts/update_gio_module_cache  | 10 +++++++---
>>>    scripts/postinst-intercepts/update_icon_cache    | 16
>>    +++++++++-------
>>>    scripts/postinst-intercepts/update_pixbuf_cache  |  5 ++++-
>>>    scripts/postinst-intercepts/update_udev_hwdb     |  6 ++++--
>>>    5 files changed, 28 insertions(+), 15 deletions(-)
>>> 
>>> diff --git a/scripts/postinst-intercepts/update_font_cache
>>    b/scripts/postinst-intercepts/update_font_cache
>>> index 3053c7065f4..954acbb41b2 100644
>>> --- a/scripts/postinst-intercepts/update_font_cache
>>> +++ b/scripts/postinst-intercepts/update_font_cache
>>> @@ -5,5 +5,7 @@
>>>    
>>>    set -e
>>>    
>>> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E
>>    ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache
>>    --sysroot=$D --system-only ${fontconfigcacheparams}
>>> -chown -R root:root $D${fontconfigcachedir}
>>> +if [ -f $D${libexecdir}/${binprefix}fc-cache ]; then
>>> +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D -E
>>    ${fontconfigcacheenv} $D${libexecdir}/${binprefix}fc-cache
>>    --sysroot=$D --system-only ${fontconfigcacheparams}
>>> +     chown -R root:root $D${fontconfigcachedir}
>>> +fi
>>> diff --git a/scripts/postinst-intercepts/update_gio_module_cache
>>    b/scripts/postinst-intercepts/update_gio_module_cache
>>> index c87fa85db9e..e8483cb0bf5 100644
>>> --- a/scripts/postinst-intercepts/update_gio_module_cache
>>> +++ b/scripts/postinst-intercepts/update_gio_module_cache
>>> @@ -5,7 +5,11 @@
>>>    
>>>    set -e
>>>    
>>> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>>    $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
>>> +if [ -f $D${libexecdir}/${binprefix}gio-querymodules ]; then
>>> +
>>> +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>>    $D${libexecdir}/${binprefix}gio-querymodules $D${libdir}/gio/modules/
>>> +
>>> +     [ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
>>> +             chown root:root $D${libdir}/gio/modules/giomodule.cache
>>> +fi
>>>    
>>> -[ ! -e $D${libdir}/gio/modules/giomodule.cache ] ||
>>> -     chown root:root $D${libdir}/gio/modules/giomodule.cache
>>> diff --git a/scripts/postinst-intercepts/update_icon_cache
>>    b/scripts/postinst-intercepts/update_icon_cache
>>> index 212209a9daf..a67ff7ef970 100644
>>> --- a/scripts/postinst-intercepts/update_icon_cache
>>> +++ b/scripts/postinst-intercepts/update_icon_cache
>>> @@ -5,12 +5,14 @@
>>>    
>>>    set -e
>>>    
>>> -# update native pixbuf loaders
>>> 
>>    -$STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
>>    --update-cache
>>> +if [ -d $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0 ]; then
>>> +     # update native pixbuf loaders
>>> +   
>>     $STAGING_DIR_NATIVE/${libdir_native}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
>>    --update-cache
>>>    
>>> -for icondir in $D/usr/share/icons/*/ ; do
>>> -    if [ -d $icondir ] ; then
>>> -        gtk-update-icon-cache -fqt  $icondir
>>> -    fi
>>> -done
>>> +     for icondir in $D/usr/share/icons/*/ ; do
>>> +         if [ -d $icondir ] ; then
>>> +             gtk-update-icon-cache -fqt  $icondir
>>> +         fi
>>> +     done
>>> +fi
>>>    
>>> diff --git a/scripts/postinst-intercepts/update_pixbuf_cache
>>    b/scripts/postinst-intercepts/update_pixbuf_cache
>>> index ea12814474e..166422e5c37 100644
>>> --- a/scripts/postinst-intercepts/update_pixbuf_cache
>>> +++ b/scripts/postinst-intercepts/update_pixbuf_cache
>>> @@ -8,6 +8,9 @@ set -e
>>>    export
>>    GDK_PIXBUF_MODULEDIR=$D${libdir}/gdk-pixbuf-2.0/2.10.0/loaders
>>>    export GDK_PIXBUF_FATAL_LOADER=1
>>>    
>>> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>>    $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
>>> +if [ -d $D${libdir}/gdk-pixbuf-2.0 ]; then
>>> +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>>    $D${libdir}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders \
>>>        >$GDK_PIXBUF_MODULEDIR/../loaders.cache && \
>>>        sed -i -e "s:$D::g" $GDK_PIXBUF_MODULEDIR/../loaders.cache
>>> +fi
>>> +
>>> diff --git a/scripts/postinst-intercepts/update_udev_hwdb
>>    b/scripts/postinst-intercepts/update_udev_hwdb
>>> index c4fb2bffcbf..7b6e38338f3 100644
>>> --- a/scripts/postinst-intercepts/update_udev_hwdb
>>> +++ b/scripts/postinst-intercepts/update_udev_hwdb
>>> @@ -5,5 +5,7 @@
>>>    
>>>    set -e
>>>    
>>> -PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>>    $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
>>> -chown root:root $D${sysconfdir}/udev/hwdb.bin
>>> +if [ -d $D/lib/udev/hwdb.d ]; then
>>> +     PSEUDO_UNLOAD=1 ${binprefix}qemuwrapper -L $D
>>    $D${libexecdir}/${binprefix}udevadm hwdb --update --root $D
>>> +     chown root:root $D${sysconfdir}/udev/hwdb.bin
>>> +fi
>> 
>> 
>>    -- 
>>    _______________________________________________
>>    Openembedded-core mailing list
>>    Openembedded-core@lists.openembedded.org
>>    <mailto:Openembedded-core@lists.openembedded.org>
>>    http://lists.openembedded.org/mailman/listinfo/openembedded-core
>> 
>
Richard Purdie June 26, 2019, 9:21 p.m.
On Wed, 2019-06-26 at 13:33 -0400, Sinan Kaya wrote:
> Here is my investigation:
> 
> The intercepts are executed on QEMU during cross-compilation using
> qemu-wrapper.
> 
> I see that the tools were built for the native architecture but
> of course they do not exist on the target image and I have no
> interest in pulling fonts and such into my image to make bitbake
> happy.
> 
> DEBUG: Collected intercepts:
>   poky/scripts/postinst-intercepts/update_font_cache
>   poky/scripts/postinst-intercepts/update_gio_module_cache
>   poky/scripts/postinst-intercepts/update_gtk_immodules_cache
>   poky/scripts/postinst-intercepts/update_icon_cache
>   poky/scripts/postinst-intercepts/update_pixbuf_cache
>   poky/scripts/postinst-intercepts/update_udev_hwdb
> 
> I also see that other intercepts are doing the same folder/executable
> checks.
> 
> What is so special about these?

Put another way, why aren't lots of people seeing failures due to this?

You're obviously doing something differently to everyone else but we're
having a hard time understanding what, or how we'd trigger the problem
you're seeing. 

We're nervous about fixing problems we don't understand, not least as
it potentially means we have a huge hole in our test matrix. Your
patches shouldn't be necessary as I understand the codebase either.

Cheers,

Richard
Sinan Kaya June 26, 2019, 10:57 p.m.
On 6/26/2019 5:21 PM, Richard Purdie wrote:
>> What is so special about these?
> Put another way, why aren't lots of people seeing failures due to this?
> 
> You're obviously doing something differently to everyone else but we're
> having a hard time understanding what, or how we'd trigger the problem
> you're seeing. 
> 
> We're nervous about fixing problems we don't understand, not least as
> it potentially means we have a huge hole in our test matrix. Your
> patches shouldn't be necessary as I understand the codebase either.

Let's see if we can peel the onion.

Can you explain how it was designed to work for cross compilation use
case where host machine architecture != target machine architecture ?

If I understand this right, all of these postinst intercepts create
a native package dependency via inherit foobar.class.

AFAIK, Native packages are built against the host machine architecture
not target architecture.

Where are the tools supposed to come from?

I can focus on details if my build actually follows the design.

As I said, I do not want these packages on my target machine but we
want to run QEMU using the tools that were compiled against target
machine in order to execute postinst intercepts.

This issue showed up in thud using core-image-minimal. It was not there
on sumo.
Alexander Kanavin June 27, 2019, 8:46 a.m.
On Thu, 27 Jun 2019 at 00:57, Sinan Kaya <Okaya@kernel.org> wrote:

> Let's see if we can peel the onion.
>
> Can you explain how it was designed to work for cross compilation use
> case where host machine architecture != target machine architecture ?
>
> If I understand this right, all of these postinst intercepts create
> a native package dependency via inherit foobar.class.
>

What makes you think so? Please point to the specific file and line where
you think this happens.


> This issue showed up in thud using core-image-minimal. It was not there
> on sumo.
>

If you can provide steps to reproduce, it could help. Please start with a
git checkout of poky thud branch.

Alex
Richard Purdie June 27, 2019, 8:57 a.m.
On Wed, 2019-06-26 at 18:57 -0400, Sinan Kaya wrote:
> On 6/26/2019 5:21 PM, Richard Purdie wrote:
> > > What is so special about these?
> > Put another way, why aren't lots of people seeing failures due to
> > this?
> > 
> > You're obviously doing something differently to everyone else but
> > we're
> > having a hard time understanding what, or how we'd trigger the
> > problem
> > you're seeing. 
> > 
> > We're nervous about fixing problems we don't understand, not least
> > as
> > it potentially means we have a huge hole in our test matrix. Your
> > patches shouldn't be necessary as I understand the codebase either.
> 
> Let's see if we can peel the onion.
> 
> Can you explain how it was designed to work for cross compilation use
> case where host machine architecture != target machine architecture ?

postinst intercepts are there to collect up certain postinst commands
and call them once rather than multiple times during rootfs generation.

There are three different cases:

a) Sometimes we can use a native tool. Here, we run the native tool
with the knowledge that the output it generates is cross compatible.

b) Sometimes we run target binaries under usermode qemu. In this case
the output is target architecture specific and we have no other way to
generate the output other than with the target binary

c) We have a cross tool which can run natively but generate target
output.

> If I understand this right, all of these postinst intercepts create
> a native package dependency via inherit foobar.class.
> 
> AFAIK, Native packages are built against the host machine
> architecture not target architecture.
> 
> Where are the tools supposed to come from?

It depends which scenario above we're dealing with. Sometimes native
recipes are used, sometimes not.

> I can focus on details if my build actually follows the design.
> 
> As I said, I do not want these packages on my target machine but we
> want to run QEMU using the tools that were compiled against target
> machine in order to execute postinst intercepts.
> 
> This issue showed up in thud using core-image-minimal. It was not
> there on sumo.

The key thing is that these intercept scripts are only run if an image
contains packages which use the postinst. If the image contains the
postinst, the scripts should run correctly.

In that context your patch is concerning as for example you're saying
you can have an image which needs the gio module cache yet it doesn't
have ${libexecdir}/${binprefix}gio-querymodules (the target binary
needed to be run under qemu to generate it). We'd define that as a bug
and you're just hiding the problem with the added conditionals. So on
that basis the patch is clearly incorrect and can't be merged as it
will hide bugs.

Cheers,

Richard
Sinan Kaya June 30, 2019, 4:56 p.m.
On 6/27/2019 4:46 AM, Alexander Kanavin wrote:
>     This issue showed up in thud using core-image-minimal. It was not there
>     on sumo.
> 
> 
> If you can provide steps to reproduce, it could help. Please start with
> a git checkout of poky thud branch.
> 

Issue in only happening on the build machine with docker using ubuntu
16.04 as a baseline.

I'm having hard-time reproducing on my development machine.

My cursory web search says that I'm not the only one hitting this issue
and people have mysteriously solved this issue.

https://github.com/riscv/meta-riscv/issues/26

The following matches what I see on the build machine.

"/update_pixbuf_cache: 8:
/home/riscv-yocto/build/tmp-glibc/work/qemuriscv64-oe-linux/core-image-full-cmdline/1.0-r0/intercept_scripts-2c03b46bc7941049a88b6c1719c35aade81b0ce5490421e7cca768bf8beb0d01

/update_pixbuf_cache: cannot create
/home//riscv-yocto/build/tmp-glibc/work/qemuriscv64-oe-linux/core-image-full-cmdline/1.0-r0/rootfs/usr/lib/gdk-pixbuf-2.0/2.10.0/loaders/../loaders.cache:
Directory nonexistent"
Richard Purdie June 30, 2019, 9:54 p.m.
On Sun, 2019-06-30 at 12:56 -0400, Sinan Kaya wrote:
> On 6/27/2019 4:46 AM, Alexander Kanavin wrote:
> >     This issue showed up in thud using core-image-minimal. It was
> > not there
> >     on sumo.
> > 
> > 
> > If you can provide steps to reproduce, it could help. Please start
> > with
> > a git checkout of poky thud branch.
> > 
> 
> Issue in only happening on the build machine with docker using ubuntu
> 16.04 as a baseline.
> 
> I'm having hard-time reproducing on my development machine.
> 
> My cursory web search says that I'm not the only one hitting this
> issue
> and people have mysteriously solved this issue.
> 
> https://github.com/riscv/meta-riscv/issues/26
> 
> The following matches what I see on the build machine.
> 
> "/update_pixbuf_cache: 8:
> /home/riscv-yocto/build/tmp-glibc/work/qemuriscv64-oe-linux/core-
> image-full-cmdline/1.0-r0/intercept_scripts-
> 2c03b46bc7941049a88b6c1719c35aade81b0ce5490421e7cca768bf8beb0d01
> 
> /update_pixbuf_cache: cannot create
> /home//riscv-yocto/build/tmp-glibc/work/qemuriscv64-oe-linux/core-
> image-full-cmdline/1.0-r0/rootfs/usr/lib/gdk-pixbuf-
> 2.0/2.10.0/loaders/../loaders.cache:
> Directory nonexistent"

Knowing which branches (sumo vs. thud) and which layers does help put a
different perspective on this issue and helps us help you!

Is this a difference in dash shell vs bash shell (e.g. for /bin/sh) on
these machines? (if I had to guess that is where I'd start)

Cheers,

Richard

>
Sinan Kaya July 1, 2019, 1:38 a.m.
On 6/30/2019 5:54 PM, richard.purdie@linuxfoundation.org wrote:
>> /update_pixbuf_cache: cannot create
>> /home//riscv-yocto/build/tmp-glibc/work/qemuriscv64-oe-linux/core-
>> image-full-cmdline/1.0-r0/rootfs/usr/lib/gdk-pixbuf-
>> 2.0/2.10.0/loaders/../loaders.cache:
>> Directory nonexistent"
> Knowing which branches (sumo vs. thud) and which layers does help put a
> different perspective on this issue and helps us help you!
> 

Sorry, I thought I mentioned this before.
Issue showed up in thud. It was not there on sumo.


> Is this a difference in dash shell vs bash shell (e.g. for /bin/sh) on
> these machines? (if I had to guess that is where I'd start)


Dash vs. bash didn't make a difference. I changed build machine to have
/bin/sh point to /bin/bash rather than /bin/dash.

The commit on github also says "it may help" with the postinst.
Apparently, the issue has been solved mysteriously for them.

One thing that issue on github mentions is that these issues were being
ignored in sumo but are marked as real failures starting from thud.