| Message ID | 20221020223004.1665512-1-raj.khem@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | rust: Disable 64bit atomics on crossbeam on riscv32 | expand |
We carry this patch, so that manual per-target fixups like this are not needed, and crossbeam's internal lists are used properly: https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/rust/rust/crossbeam_atomic.patch?h=master-next&id=f4fe7cdaa7fc5785c55bd507c897f4d25d43cffa Is it not working somehow? Does the list itself need to be fixed? Alex On Fri, 21 Oct 2022 at 00:30, Khem Raj <raj.khem@gmail.com> wrote: > > riscv32 builds fail since AtomicI64 is not available in sync::atomic for > this architecture. Therefore it needs to be opted out and the way out is > to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS > before exporting it so its better to piggy back on the variable its > using which is RUST_DEBUG_REMAP > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > --- > meta/recipes-devtools/rust/rust_1.64.0.bb | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/recipes-devtools/rust/rust_1.64.0.bb b/meta/recipes-devtools/rust/rust_1.64.0.bb > index 1f9dbd3cce..b05f188d9c 100644 > --- a/meta/recipes-devtools/rust/rust_1.64.0.bb > +++ b/meta/recipes-devtools/rust/rust_1.64.0.bb > @@ -9,6 +9,7 @@ FILES:${PN}-dev = "" > > # Used by crossbeam_atomic.patch > export TARGET_VENDOR > +RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64" > > do_compile () { > rust_runx build --stage 2 > -- > 2.38.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#172006): https://lists.openembedded.org/g/openembedded-core/message/172006 > Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Thu, Oct 20, 2022 at 11:04 PM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > We carry this patch, so that manual per-target fixups like this are > not needed, and crossbeam's internal lists are used properly: > > https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/rust/rust/crossbeam_atomic.patch?h=master-next&id=f4fe7cdaa7fc5785c55bd507c897f4d25d43cffa > > Is it not working somehow? Does the list itself need to be fixed? > This patch is doing something else where its mapping and broadening the architecture to match crossbeam's understanding of it. Here issue is new where it does not yet realize that some architectures do not have 64bit atomics implemented. > Alex > > On Fri, 21 Oct 2022 at 00:30, Khem Raj <raj.khem@gmail.com> wrote: > > > > riscv32 builds fail since AtomicI64 is not available in sync::atomic for > > this architecture. Therefore it needs to be opted out and the way out is > > to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS > > before exporting it so its better to piggy back on the variable its > > using which is RUST_DEBUG_REMAP > > > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > > --- > > meta/recipes-devtools/rust/rust_1.64.0.bb | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/meta/recipes-devtools/rust/rust_1.64.0.bb b/meta/recipes-devtools/rust/rust_1.64.0.bb > > index 1f9dbd3cce..b05f188d9c 100644 > > --- a/meta/recipes-devtools/rust/rust_1.64.0.bb > > +++ b/meta/recipes-devtools/rust/rust_1.64.0.bb > > @@ -9,6 +9,7 @@ FILES:${PN}-dev = "" > > > > # Used by crossbeam_atomic.patch > > export TARGET_VENDOR > > +RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64" > > > > do_compile () { > > rust_runx build --stage 2 > > -- > > 2.38.1 > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#172006): https://lists.openembedded.org/g/openembedded-core/message/172006 > > Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > >
On Fri, 21 Oct 2022 at 16:05, Khem Raj <raj.khem@gmail.com> wrote: > This patch is doing something else where its mapping and broadening > the architecture to match crossbeam's understanding of it. > Here issue is new where it does not yet realize that some > architectures do not have 64bit atomics implemented. Here's crossbeam's own list: https://github.com/crossbeam-rs/crossbeam/blob/master/no_atomic.rs You can see that riscv32 is in NO_ATOMIC_64, and crossbeam_atomic.patch ensures we use the list, so why do we need this change? Alex
On Fri, Oct 21, 2022 at 8:41 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Fri, 21 Oct 2022 at 16:05, Khem Raj <raj.khem@gmail.com> wrote: > > This patch is doing something else where its mapping and broadening > > the architecture to match crossbeam's understanding of it. > > Here issue is new where it does not yet realize that some > > architectures do not have 64bit atomics implemented. > > Here's crossbeam's own list: > > https://github.com/crossbeam-rs/crossbeam/blob/master/no_atomic.rs > > You can see that riscv32 is in NO_ATOMIC_64, and > crossbeam_atomic.patch ensures we use the list, so why do we need this > change? Looking briefly over the code, my educated guess is that target names are still not matching rusts tuple uses riscv32gc but we use riscv32 > > Alex
On Fri, 21 Oct 2022 at 18:07, Khem Raj <raj.khem@gmail.com> wrote: > Looking briefly over the code, my educated guess is that target names > are still not matching > rusts tuple uses riscv32gc but we use riscv32 I believe we have a remapping in meta/lib/oe/rust.py, can you check if that works? Alex
On Fri, Oct 21, 2022 at 9:19 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Fri, 21 Oct 2022 at 18:07, Khem Raj <raj.khem@gmail.com> wrote: > > Looking briefly over the code, my educated guess is that target names > > are still not matching > > rusts tuple uses riscv32gc but we use riscv32 > > I believe we have a remapping in meta/lib/oe/rust.py, can you check if > that works? This is mapping host uname, it wont work. > > Alex
On Fri, 21 Oct 2022 at 19:00, Khem Raj <raj.khem@gmail.com> wrote: > > On Fri, 21 Oct 2022 at 18:07, Khem Raj <raj.khem@gmail.com> wrote: > > > Looking briefly over the code, my educated guess is that target names > > > are still not matching > > > rusts tuple uses riscv32gc but we use riscv32 > > > > I believe we have a remapping in meta/lib/oe/rust.py, can you check if > > that works? > > This is mapping host uname, it wont work. No, it's mapping the targets too. E.g. in rust-common.bbclass: # Responsible for taking Yocto triples and converting it to Rust triples def rust_base_triple(d, thing): ''' Mangle bitbake's *_SYS into something that rust might support (see rust/mk/cfg/* for a list) Note that os is assumed to be some linux form ''' # The llvm-target for armv7 is armv7-unknown-linux-gnueabihf if d.getVar('{}_ARCH'.format(thing)) == d.getVar('TARGET_ARCH') and target_is_armv7(d): arch = "armv7" else: arch = oe.rust.arch_to_rust_arch(d.getVar('{}_ARCH'.format(thing))) Alex
On Fri, Oct 21, 2022 at 10:12 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Fri, 21 Oct 2022 at 19:00, Khem Raj <raj.khem@gmail.com> wrote: > > > On Fri, 21 Oct 2022 at 18:07, Khem Raj <raj.khem@gmail.com> wrote: > > > > Looking briefly over the code, my educated guess is that target names > > > > are still not matching > > > > rusts tuple uses riscv32gc but we use riscv32 > > > > > > I believe we have a remapping in meta/lib/oe/rust.py, can you check if > > > that works? > > > > This is mapping host uname, it wont work. > > No, it's mapping the targets too. E.g. in rust-common.bbclass: > > # Responsible for taking Yocto triples and converting it to Rust triples > def rust_base_triple(d, thing): > ''' > Mangle bitbake's *_SYS into something that rust might support (see > rust/mk/cfg/* for a list) > > Note that os is assumed to be some linux form > ''' > > # The llvm-target for armv7 is armv7-unknown-linux-gnueabihf > if d.getVar('{}_ARCH'.format(thing)) == d.getVar('TARGET_ARCH') > and target_is_armv7(d): > arch = "armv7" > else: > arch = oe.rust.arch_to_rust_arch(d.getVar('{}_ARCH'.format(thing))) > anyway, the builds failed at exactly the same point so as I sensed it does not work. > Alex
On Fri, 21 Oct 2022 at 19:16, Khem Raj <raj.khem@gmail.com> wrote: > anyway, the builds failed at exactly the same point so as I sensed it > does not work. You need to find out why though. Upstream maintains a list, and we need to use it, and not duplicate it piecemeal. Alex
On Fri, Oct 21, 2022 at 10:28 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Fri, 21 Oct 2022 at 19:16, Khem Raj <raj.khem@gmail.com> wrote: > > > anyway, the builds failed at exactly the same point so as I sensed it > > does not work. > > You need to find out why though. Upstream maintains a list, and we > need to use it, and not duplicate it piecemeal. It needs to be mapped or added. I have sent a patch to add OE's understanding of target arch. > > Alex
Ok, I'd like to understand where the failure is exactly. I just ran 'bitbake rust' with poky master (rust 1.63) for qemuriscv32 machine, and it succeeded. How can I see the fail? Did this become broken with a 1.64.0 upgrade in your branch? (which I believe hasn't yet been through the autobuilder, and so might show similar fails for other targets without atomic_64 support) Alex On Fri, 21 Oct 2022 at 00:30, Khem Raj <raj.khem@gmail.com> wrote: > > riscv32 builds fail since AtomicI64 is not available in sync::atomic for > this architecture. Therefore it needs to be opted out and the way out is > to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS > before exporting it so its better to piggy back on the variable its > using which is RUST_DEBUG_REMAP > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > --- > meta/recipes-devtools/rust/rust_1.64.0.bb | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/recipes-devtools/rust/rust_1.64.0.bb b/meta/recipes-devtools/rust/rust_1.64.0.bb > index 1f9dbd3cce..b05f188d9c 100644 > --- a/meta/recipes-devtools/rust/rust_1.64.0.bb > +++ b/meta/recipes-devtools/rust/rust_1.64.0.bb > @@ -9,6 +9,7 @@ FILES:${PN}-dev = "" > > # Used by crossbeam_atomic.patch > export TARGET_VENDOR > +RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64" > > do_compile () { > rust_runx build --stage 2 > -- > 2.38.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#172006): https://lists.openembedded.org/g/openembedded-core/message/172006 > Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Ok, I have looked into this properly now. So: - the fail started to occur in 1.64 because crossbeam-utils is now a dependency of rust core (hard fail) and not just rustfmt (soft fail) - the correct way to address it is indeed to map oe arch (riscv32) to rust arch (riscv32gc) via meta/lib/oe/rust.py, which works globally across all recipes where this may occur. - however doing that exposed confusion about how we map arches, as there is actually a *third* arch involved (the internal llvm one listed in custom target definition), which too needs to be correctly mapped from oe arch so I fixed that as well. None of the three arch sets match each other exactly, unfortunately. Patches are coming shortly. Alex On Sat, 22 Oct 2022 at 21:56, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > Ok, I'd like to understand where the failure is exactly. I just ran > 'bitbake rust' with poky master (rust 1.63) for qemuriscv32 machine, > and it succeeded. How can I see the fail? Did this become broken with > a 1.64.0 upgrade in your branch? (which I believe hasn't yet been > through the autobuilder, and so might show similar fails for other > targets without atomic_64 support) > > Alex > > On Fri, 21 Oct 2022 at 00:30, Khem Raj <raj.khem@gmail.com> wrote: > > > > riscv32 builds fail since AtomicI64 is not available in sync::atomic for > > this architecture. Therefore it needs to be opted out and the way out is > > to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS > > before exporting it so its better to piggy back on the variable its > > using which is RUST_DEBUG_REMAP > > > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > > --- > > meta/recipes-devtools/rust/rust_1.64.0.bb | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/meta/recipes-devtools/rust/rust_1.64.0.bb b/meta/recipes-devtools/rust/rust_1.64.0.bb > > index 1f9dbd3cce..b05f188d9c 100644 > > --- a/meta/recipes-devtools/rust/rust_1.64.0.bb > > +++ b/meta/recipes-devtools/rust/rust_1.64.0.bb > > @@ -9,6 +9,7 @@ FILES:${PN}-dev = "" > > > > # Used by crossbeam_atomic.patch > > export TARGET_VENDOR > > +RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64" > > > > do_compile () { > > rust_runx build --stage 2 > > -- > > 2.38.1 > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#172064): https://lists.openembedded.org/g/openembedded-core/message/172064 > Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Tue, Oct 25, 2022 at 9:15 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > Ok, I have looked into this properly now. So: > > - the fail started to occur in 1.64 because crossbeam-utils is now a > dependency of rust core (hard fail) and not just rustfmt (soft fail) > - the correct way to address it is indeed to map oe arch (riscv32) to > rust arch (riscv32gc) via meta/lib/oe/rust.py, which works globally > across all recipes where this may occur. > - however doing that exposed confusion about how we map arches, as > there is actually a *third* arch involved (the internal llvm one > listed in custom target definition), which too needs to be correctly > mapped from oe arch so I fixed that as well. > > None of the three arch sets match each other exactly, unfortunately. Right. > > Patches are coming shortly. > Thanks, I think we can also unbolt the workaround in librsvg as well. diff --git a/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb index fc52ae61c5..daf60217d4 100644 --- a/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb +++ b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb @@ -35,7 +35,6 @@ export RUST_TARGET = "${RUST_HOST_SYS}" RUSTFLAGS:append:mips = " --cfg crossbeam_no_atomic_64" RUSTFLAGS:append:mipsel = " --cfg crossbeam_no_atomic_64" RUSTFLAGS:append:powerpc = " --cfg crossbeam_no_atomic_64" -RUSTFLAGS:append:riscv32 = " --cfg crossbeam_no_atomic_64" CARGO_DISABLE_BITBAKE_VENDORING = "1" do_configure[postfuncs] += "cargo_common_do_configure" > Alex > > On Sat, 22 Oct 2022 at 21:56, Alexander Kanavin via > lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> > wrote: > > > > Ok, I'd like to understand where the failure is exactly. I just ran > > 'bitbake rust' with poky master (rust 1.63) for qemuriscv32 machine, > > and it succeeded. How can I see the fail? Did this become broken with > > a 1.64.0 upgrade in your branch? (which I believe hasn't yet been > > through the autobuilder, and so might show similar fails for other > > targets without atomic_64 support) > > > > Alex > > > > On Fri, 21 Oct 2022 at 00:30, Khem Raj <raj.khem@gmail.com> wrote: > > > > > > riscv32 builds fail since AtomicI64 is not available in sync::atomic for > > > this architecture. Therefore it needs to be opted out and the way out is > > > to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS > > > before exporting it so its better to piggy back on the variable its > > > using which is RUST_DEBUG_REMAP > > > > > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > > > --- > > > meta/recipes-devtools/rust/rust_1.64.0.bb | 1 + > > > 1 file changed, 1 insertion(+) > > > > > > diff --git a/meta/recipes-devtools/rust/rust_1.64.0.bb b/meta/recipes-devtools/rust/rust_1.64.0.bb > > > index 1f9dbd3cce..b05f188d9c 100644 > > > --- a/meta/recipes-devtools/rust/rust_1.64.0.bb > > > +++ b/meta/recipes-devtools/rust/rust_1.64.0.bb > > > @@ -9,6 +9,7 @@ FILES:${PN}-dev = "" > > > > > > # Used by crossbeam_atomic.patch > > > export TARGET_VENDOR > > > +RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64" > > > > > > do_compile () { > > > rust_runx build --stage 2 > > > -- > > > 2.38.1 > > > > > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#172064): https://lists.openembedded.org/g/openembedded-core/message/172064 > > Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > >
On Wed, 26 Oct 2022 at 02:46, Khem Raj <raj.khem@gmail.com> wrote: > Thanks, I think we can also unbolt the workaround in librsvg as well. > > diff --git a/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > index fc52ae61c5..daf60217d4 100644 > --- a/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > +++ b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > @@ -35,7 +35,6 @@ export RUST_TARGET = "${RUST_HOST_SYS}" > RUSTFLAGS:append:mips = " --cfg crossbeam_no_atomic_64" > RUSTFLAGS:append:mipsel = " --cfg crossbeam_no_atomic_64" > RUSTFLAGS:append:powerpc = " --cfg crossbeam_no_atomic_64" > -RUSTFLAGS:append:riscv32 = " --cfg crossbeam_no_atomic_64" Not yet. librsvg carries its own copy of crossbeam which is not patched to use the provided standard lists. So we should first merge some variant of crossbeam_atomic.patch upstream. My previous attempt went nowhere: https://github.com/crossbeam-rs/crossbeam/pull/751 Alex
On Wed, Oct 26, 2022 at 12:58 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > > On Wed, 26 Oct 2022 at 02:46, Khem Raj <raj.khem@gmail.com> wrote: > > Thanks, I think we can also unbolt the workaround in librsvg as well. > > > > diff --git a/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > > b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > > index fc52ae61c5..daf60217d4 100644 > > --- a/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > > +++ b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb > > @@ -35,7 +35,6 @@ export RUST_TARGET = "${RUST_HOST_SYS}" > > RUSTFLAGS:append:mips = " --cfg crossbeam_no_atomic_64" > > RUSTFLAGS:append:mipsel = " --cfg crossbeam_no_atomic_64" > > RUSTFLAGS:append:powerpc = " --cfg crossbeam_no_atomic_64" > > -RUSTFLAGS:append:riscv32 = " --cfg crossbeam_no_atomic_64" > > Not yet. librsvg carries its own copy of crossbeam which is not > patched to use the provided standard lists. right saw that > > So we should first merge some variant of crossbeam_atomic.patch > upstream. My previous attempt went nowhere: > https://github.com/crossbeam-rs/crossbeam/pull/751 > > Alex
diff --git a/meta/recipes-devtools/rust/rust_1.64.0.bb b/meta/recipes-devtools/rust/rust_1.64.0.bb index 1f9dbd3cce..b05f188d9c 100644 --- a/meta/recipes-devtools/rust/rust_1.64.0.bb +++ b/meta/recipes-devtools/rust/rust_1.64.0.bb @@ -9,6 +9,7 @@ FILES:${PN}-dev = "" # Used by crossbeam_atomic.patch export TARGET_VENDOR +RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64" do_compile () { rust_runx build --stage 2
riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP Signed-off-by: Khem Raj <raj.khem@gmail.com> --- meta/recipes-devtools/rust/rust_1.64.0.bb | 1 + 1 file changed, 1 insertion(+)