diff mbox series

[kirkstone,13/15] gobject-introspection-data: Disable cache for g-ir-scanner

Message ID bb1dc2bbfe268e73008dc37d6b1f83a861b48915.1658882276.git.steve@sakoman.com
State New, archived
Headers show
Series [kirkstone,01/15] lua: Backport fix for CVE-2022-33099 | expand

Commit Message

Steve Sakoman July 27, 2022, 12:40 a.m. UTC 
From: Tom Hochstein <tom.hochstein@nxp.com>

An intermittent failure occurs in libical-native do_compile:

| Traceback (most recent call last):
|   File "/.../build/tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 813, in move
|     os.rename(src, real_dst)
| OSError: [Errno 18] Invalid cross-device link: '/tmp/g-ir-scanner-cache-adxo_2bq' -> '/home/bamboo/.cache/g-ir-scanner/bab9a83d2cd93e62ed005a2c1d4f89ae75c67251'
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/bin/g-ir-scanner", line 99, in <module>
|     sys.exit(scanner_main(sys.argv))
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/scannermain.py", line 590, in scanner_main
|     transformer = create_transformer(namespace, options)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/scannermain.py", line 409, in create_transformer
|     transformer.register_include(include_obj)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py", line 140, in register_include
|     self._parse_include(filename)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py", line 230, in _parse_include
|     self._parse_include(dep_filename)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py", line 225, in _parse_include
|     self._cachestore.store(filename, parser)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/cachestore.py", line 153, in store
|     shutil.move(tmp_filename, store_filename)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 833, in move
|     copy_function(src, real_dst)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 435, in copy2
|     copystat(src, dst, follow_symlinks=follow_symlinks)
|   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 374, in copystat
|     lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
| FileNotFoundError: [Errno 2] No such file or directory

A similar issue is described in a fix for gobject-introspection-native.

https://github.com/openembedded/openembedded-core/commit/d3c48ff7d19e86b2338b1778f9563969bba3d336

The problem was fixed there by setting the environment variable
GI_SCANNER_DISABLE_CACHE to disable the use of $HOME/.cache.

Extend the fix to users of gobject-instropection by promoting the fix
to the bbclass.

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 35d5f707f6bb2ce5e9ab908e66e1ea9eeac754b1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes/gobject-introspection-data.bbclass              | 5 +++++
 .../gobject-introspection/gobject-introspection_1.72.0.bb    | 3 ---
 2 files changed, 5 insertions(+), 3 deletions(-)

Comments

Chuck Wolber July 28, 2022, 5:55 p.m. UTC | #1 
FWIW, the gobject-introspection-native recipe was one of the "canaries in
the mine" that reliably caused an image build to die while I was actively
encountering this GCC10 compiler issue in Hardknott:

https://www.mail-archive.com/yocto@lists.yoctoproject.org/msg06163.html
https://lists.yoctoproject.org/g/yocto/message/53971

Upon re-reading my post, I regret that I did not mention this fact. The
production image recipe I was working on at the time did not happen to
involve gobject-introspection-native, so I overlooked it. Our other image
recipes *DID* invoke gobject-introspection-native, but those were not as
immediately important at the time, and I was trying to keep the problem
description as focused as possible.

I do not expect this reply to result in anything materially relevant to the
problem (it still exists, but we use buildtools to get around it) other
than to add more data that may be relevant to someone googling something
similar in the future.

..Ch:W..

On Tue, Jul 26, 2022 at 5:41 PM Steve Sakoman <steve@sakoman.com> wrote:

> From: Tom Hochstein <tom.hochstein@nxp.com>
>
> An intermittent failure occurs in libical-native do_compile:
>
> | Traceback (most recent call last):
> |   File
> "/.../build/tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py",
> line 813, in move
> |     os.rename(src, real_dst)
> | OSError: [Errno 18] Invalid cross-device link:
> '/tmp/g-ir-scanner-cache-adxo_2bq' ->
> '/home/bamboo/.cache/g-ir-scanner/bab9a83d2cd93e62ed005a2c1d4f89ae75c67251'
> |
> | During handling of the above exception, another exception occurred:
> |
> | Traceback (most recent call last):
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/bin/g-ir-scanner",
> line 99, in <module>
> |     sys.exit(scanner_main(sys.argv))
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/scannermain.py",
> line 590, in scanner_main
> |     transformer = create_transformer(namespace, options)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/scannermain.py",
> line 409, in create_transformer
> |     transformer.register_include(include_obj)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py",
> line 140, in register_include
> |     self._parse_include(filename)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py",
> line 230, in _parse_include
> |     self._parse_include(dep_filename)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py",
> line 225, in _parse_include
> |     self._cachestore.store(filename, parser)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/cachestore.py",
> line 153, in store
> |     shutil.move(tmp_filename, store_filename)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py",
> line 833, in move
> |     copy_function(src, real_dst)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py",
> line 435, in copy2
> |     copystat(src, dst, follow_symlinks=follow_symlinks)
> |   File
> "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py",
> line 374, in copystat
> |     lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
> | FileNotFoundError: [Errno 2] No such file or directory
>
> A similar issue is described in a fix for gobject-introspection-native.
>
>
> https://github.com/openembedded/openembedded-core/commit/d3c48ff7d19e86b2338b1778f9563969bba3d336
>
> The problem was fixed there by setting the environment variable
> GI_SCANNER_DISABLE_CACHE to disable the use of $HOME/.cache.
>
> Extend the fix to users of gobject-instropection by promoting the fix
> to the bbclass.
>
> Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
> (cherry picked from commit 35d5f707f6bb2ce5e9ab908e66e1ea9eeac754b1)
> Signed-off-by: Steve Sakoman <steve@sakoman.com>
> ---
>  meta/classes/gobject-introspection-data.bbclass              | 5 +++++
>  .../gobject-introspection/gobject-introspection_1.72.0.bb    | 3 ---
>  2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes/gobject-introspection-data.bbclass
> b/meta/classes/gobject-introspection-data.bbclass
> index 2ef684626a..d90cdb4839 100644
> --- a/meta/classes/gobject-introspection-data.bbclass
> +++ b/meta/classes/gobject-introspection-data.bbclass
> @@ -5,3 +5,8 @@
>  # so that qemu use can be avoided when necessary.
>  GI_DATA_ENABLED ?= "${@bb.utils.contains('DISTRO_FEATURES',
> 'gobject-introspection-data', \
>                        bb.utils.contains('MACHINE_FEATURES',
> 'qemu-usermode', 'True', 'False', d), 'False', d)}"
> +
> +do_compile:prepend() {
> +    # This prevents g-ir-scanner from writing cache data to $HOME
> +    export GI_SCANNER_DISABLE_CACHE=1
> +}
> diff --git a/meta/recipes-gnome/gobject-introspection/
> gobject-introspection_1.72.0.bb
> b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
> index 355e77d107..9a47e908b7 100644
> --- a/meta/recipes-gnome/gobject-introspection/
> gobject-introspection_1.72.0.bb
> +++ b/meta/recipes-gnome/gobject-introspection/
> gobject-introspection_1.72.0.bb
> @@ -113,9 +113,6 @@ EOF
>  }
>
>  do_compile:prepend() {
> -        # This prevents g-ir-scanner from writing cache data to $HOME
> -        export GI_SCANNER_DISABLE_CACHE=1
> -
>          # Needed to run g-ir unit tests, which won't be able to find the
> built libraries otherwise
>          export GIR_EXTRA_LIBS_PATH=$B/.libs
>  }
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#168536):
> https://lists.openembedded.org/g/openembedded-core/message/168536
> Mute This Topic: https://lists.openembedded.org/mt/92640670/894569
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> chuckwolber@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Steve Sakoman July 28, 2022, 6:37 p.m. UTC | #2 
On Thu, Jul 28, 2022 at 7:55 AM Chuck Wolber <chuckwolber@gmail.com> wrote:
>
> FWIW, the gobject-introspection-native recipe was one of the "canaries in the mine" that reliably caused an image build to die while I was actively encountering this GCC10 compiler issue in Hardknott:

Have you encountered this issue with Kirkstone?

Steve

>
> https://www.mail-archive.com/yocto@lists.yoctoproject.org/msg06163.html
> https://lists.yoctoproject.org/g/yocto/message/53971
>
> Upon re-reading my post, I regret that I did not mention this fact. The production image recipe I was working on at the time did not happen to involve gobject-introspection-native, so I overlooked it. Our other image recipes *DID* invoke gobject-introspection-native, but those were not as immediately important at the time, and I was trying to keep the problem description as focused as possible.
>
> I do not expect this reply to result in anything materially relevant to the problem (it still exists, but we use buildtools to get around it) other than to add more data that may be relevant to someone googling something similar in the future.
>
> ..Ch:W..
>
> On Tue, Jul 26, 2022 at 5:41 PM Steve Sakoman <steve@sakoman.com> wrote:
>>
>> From: Tom Hochstein <tom.hochstein@nxp.com>
>>
>> An intermittent failure occurs in libical-native do_compile:
>>
>> | Traceback (most recent call last):
>> |   File "/.../build/tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 813, in move
>> |     os.rename(src, real_dst)
>> | OSError: [Errno 18] Invalid cross-device link: '/tmp/g-ir-scanner-cache-adxo_2bq' -> '/home/bamboo/.cache/g-ir-scanner/bab9a83d2cd93e62ed005a2c1d4f89ae75c67251'
>> |
>> | During handling of the above exception, another exception occurred:
>> |
>> | Traceback (most recent call last):
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/pkgconfig/../../../usr/bin/g-ir-scanner", line 99, in <module>
>> |     sys.exit(scanner_main(sys.argv))
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/scannermain.py", line 590, in scanner_main
>> |     transformer = create_transformer(namespace, options)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/scannermain.py", line 409, in create_transformer
>> |     transformer.register_include(include_obj)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py", line 140, in register_include
>> |     self._parse_include(filename)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py", line 230, in _parse_include
>> |     self._parse_include(dep_filename)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/transformer.py", line 225, in _parse_include
>> |     self._cachestore.store(filename, parser)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/gobject-introspection/giscanner/cachestore.py", line 153, in store
>> |     shutil.move(tmp_filename, store_filename)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 833, in move
>> |     copy_function(src, real_dst)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 435, in copy2
>> |     copystat(src, dst, follow_symlinks=follow_symlinks)
>> |   File "/.../tmp/work/x86_64-linux/libical-native/3.0.14-r0/recipe-sysroot-native/usr/lib/python3.10/shutil.py", line 374, in copystat
>> |     lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns),
>> | FileNotFoundError: [Errno 2] No such file or directory
>>
>> A similar issue is described in a fix for gobject-introspection-native.
>>
>> https://github.com/openembedded/openembedded-core/commit/d3c48ff7d19e86b2338b1778f9563969bba3d336
>>
>> The problem was fixed there by setting the environment variable
>> GI_SCANNER_DISABLE_CACHE to disable the use of $HOME/.cache.
>>
>> Extend the fix to users of gobject-instropection by promoting the fix
>> to the bbclass.
>>
>> Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
>> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
>> (cherry picked from commit 35d5f707f6bb2ce5e9ab908e66e1ea9eeac754b1)
>> Signed-off-by: Steve Sakoman <steve@sakoman.com>
>> ---
>>  meta/classes/gobject-introspection-data.bbclass              | 5 +++++
>>  .../gobject-introspection/gobject-introspection_1.72.0.bb    | 3 ---
>>  2 files changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/classes/gobject-introspection-data.bbclass b/meta/classes/gobject-introspection-data.bbclass
>> index 2ef684626a..d90cdb4839 100644
>> --- a/meta/classes/gobject-introspection-data.bbclass
>> +++ b/meta/classes/gobject-introspection-data.bbclass
>> @@ -5,3 +5,8 @@
>>  # so that qemu use can be avoided when necessary.
>>  GI_DATA_ENABLED ?= "${@bb.utils.contains('DISTRO_FEATURES', 'gobject-introspection-data', \
>>                        bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'True', 'False', d), 'False', d)}"
>> +
>> +do_compile:prepend() {
>> +    # This prevents g-ir-scanner from writing cache data to $HOME
>> +    export GI_SCANNER_DISABLE_CACHE=1
>> +}
>> diff --git a/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
>> index 355e77d107..9a47e908b7 100644
>> --- a/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
>> +++ b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
>> @@ -113,9 +113,6 @@ EOF
>>  }
>>
>>  do_compile:prepend() {
>> -        # This prevents g-ir-scanner from writing cache data to $HOME
>> -        export GI_SCANNER_DISABLE_CACHE=1
>> -
>>          # Needed to run g-ir unit tests, which won't be able to find the built libraries otherwise
>>          export GIR_EXTRA_LIBS_PATH=$B/.libs
>>  }
>> --
>> 2.25.1
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#168536): https://lists.openembedded.org/g/openembedded-core/message/168536
>> Mute This Topic: https://lists.openembedded.org/mt/92640670/894569
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [chuckwolber@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
>
> --
> "Perfection must be reached by degrees; she requires the slow hand of time." - Voltaire
Chuck Wolber July 28, 2022, 6:52 p.m. UTC | #3 
On Thu, Jul 28, 2022 at 11:37 AM Steve Sakoman <steve@sakoman.com> wrote:

> On Thu, Jul 28, 2022 at 7:55 AM Chuck Wolber <chuckwolber@gmail.com>
> wrote:
> >
> > FWIW, the gobject-introspection-native recipe was one of the "canaries
> in the mine" that reliably caused an image build to die while I was
> actively encountering this GCC10 compiler issue in Hardknott:
>
> Have you encountered this issue with Kirkstone?
>

We have not tried yet. We are debating the Hardknott to Kirkstone upgrade
question right now.

Our build is complex and 2.6 (Thud) -> 3.3 (Hardknott) took two months of
very intensive work and then six months of stabilization. Granted python2
-> python3 was part of that, so a big chunk of the stabilization damage was
self-inflicted. I do not expect anything as major for Hardknott ->
Kirkstone, but I am also not naïve either - all blockpoint upgrades are
painful in their own way...

..Ch:W..
diff mbox series

Patch

diff --git a/meta/classes/gobject-introspection-data.bbclass b/meta/classes/gobject-introspection-data.bbclass
index 2ef684626a..d90cdb4839 100644
--- a/meta/classes/gobject-introspection-data.bbclass
+++ b/meta/classes/gobject-introspection-data.bbclass
@@ -5,3 +5,8 @@ 
 # so that qemu use can be avoided when necessary.
 GI_DATA_ENABLED ?= "${@bb.utils.contains('DISTRO_FEATURES', 'gobject-introspection-data', \
                       bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'True', 'False', d), 'False', d)}"
+
+do_compile:prepend() {
+    # This prevents g-ir-scanner from writing cache data to $HOME
+    export GI_SCANNER_DISABLE_CACHE=1
+}
diff --git a/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
index 355e77d107..9a47e908b7 100644
--- a/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
+++ b/meta/recipes-gnome/gobject-introspection/gobject-introspection_1.72.0.bb
@@ -113,9 +113,6 @@  EOF
 }
 
 do_compile:prepend() {
-        # This prevents g-ir-scanner from writing cache data to $HOME
-        export GI_SCANNER_DISABLE_CACHE=1
-
         # Needed to run g-ir unit tests, which won't be able to find the built libraries otherwise
         export GIR_EXTRA_LIBS_PATH=$B/.libs
 }