| Message ID | 20240118105723.904877-1-Yash.Shinde@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | rust: Enable rust oe-selftest | expand |
On 2024-01-18 5:57 a.m., Yash.Shinde@windriver.com wrote: > From: Yash Shinde<Yash.Shinde@windriver.com> > > Enable rust oe-selftest for rust 1.74.1 version and > add target_cfg.patch to handle target configurations > for custom targets. Hi Yashe, Yay, getting the rust test suite working again great news! We talked about this in today's YP bug review and decided NOT to merge this until after the M2 build is taken care of over the next few days. That gives me a chance to review the commit and ask some questions! ;-) First, your commit log should be expanded to explain what has changed at a high level to someone who is just reading "git log". What you have is a good start but I'd like more info on what changed in: meta/lib/oeqa/selftest/cases/rust.py There are lots of tests added to the exclude list and some seem to have just moved. Please explain why you did this and if there is any way to avoid the "noise" of moved tests in future updates. Could you also report on the number of newly disabled tests and try to summarize in the commit log what these tests are doing and Ah, I see what you have done: 1. sort the exclude list 2. add the newly excluded tests. Please do that in separate commits so that it's easier to understand. This will reduce the size of the "Enable rust oe-selftest" diff for meta/lib/oeqa/selftest/cases/rust.py from: | 103 ++++++++----- to: | 37 +++- and let people easily see that most of the newly excluded tests are in tests/codegen . That reminds me, do we really need to list directories in that exclude list or only list .rs files? If I save the list to /tmp/jj, you can see that we could reduce the list by 34 lines: ❯ rg "\.rs" /tmp/jj | wc -l 203 ❯ rg -v "\.rs" /tmp/jj | wc -l 34 If we can do that, again do it as a separate commit since it should not affect the outcome of the test. Also this backport link: https://github.com/rust-lang/rust/pull/119619/commits is actually 3 separate commits so it might be better to apply those individually so that we can know when they can be dropped by running: $ git tag --contains <commit-id> on each commit Also in the patches, please add you Signed-Off-By line but keep the upstream author's S-O-B line. For anyone interested we did talk about the number of Rust tests that we are excluding and of course our goal is to work with the upstream Rust community to reduce the number of excluded tests over the coming months but for now, it's just nice to see the test working again! If you update these things convince yourself that the changes are functionally identical, I don't think there's any need to re-do all the testing that you have done. Thanks Yash, ../Randy > > The testing is done on arm32, arm64, mips64, x86 and x86_64 targets. > > Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com> > --- > meta/lib/oeqa/selftest/cases/rust.py | 103 ++++++++----- > .../rust/files/target_cfg.patch | 144 ++++++++++++++++++ > meta/recipes-devtools/rust/rust-source.inc | 1 + > 3 files changed, 214 insertions(+), 34 deletions(-) > create mode 100644 meta/recipes-devtools/rust/files/target_cfg.patch > > diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py > index 6dbc517006..216c73e5d1 100644 > --- a/meta/lib/oeqa/selftest/cases/rust.py > +++ b/meta/lib/oeqa/selftest/cases/rust.py > @@ -40,7 +40,7 @@ def parse_results(filename): > class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > def test_rust(self, *args, **kwargs): > # Disable Rust Oe-selftest > - self.skipTest("The Rust Oe-selftest is disabled.") > + #self.skipTest("The Rust Oe-selftest is disabled.") > > # build remote-test-server before image build > recipe = "rust" > @@ -73,38 +73,71 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > 'src/librustdoc', > 'src/rustdoc-json-types', > 'src/tools/compiletest/src/common.rs', > + 'src/tools/jsondoclint', > 'src/tools/lint-docs', > + 'src/tools/replace-version-placeholder', > 'src/tools/rust-analyzer', > 'src/tools/rustdoc-themes', > 'src/tools/tidy', > 'tests/assembly/asm/aarch64-outline-atomics.rs', > + 'tests/codegen/abi-efiapi.rs', > + 'tests/codegen/abi-efiapi.rs', > 'tests/codegen/abi-main-signature-32bit-c-int.rs', > 'tests/codegen/abi-repr-ext.rs', > + 'tests/codegen/abi-sysv64.rs', > 'tests/codegen/abi-x86-interrupt.rs', > + 'tests/codegen/align-byval.rs', > + 'tests/codegen/align-fn.rs', > + 'tests/codegen/asm-powerpc-clobbers.rs', > + 'tests/codegen/async-fn-debug-awaitee-field.rs', > + 'tests/codegen/binary-search-index-no-bound-check.rs', > 'tests/codegen/branch-protection.rs', > + 'tests/codegen/call-metadata.rs', > 'tests/codegen/catch-unwind.rs', > 'tests/codegen/cf-protection.rs', > + 'tests/codegen/debug-column.rs', > + 'tests/codegen/debug-limited.rs', > + 'tests/codegen/debuginfo-generic-closure-env-names.rs', > + 'tests/codegen/drop.rs', > + 'tests/codegen/dst-vtable-align-nonzero.rs', > + 'tests/codegen/enable-lto-unit-splitting.rs', > 'tests/codegen/enum-bounds-check-derived-idx.rs', > + 'tests/codegen/enum/enum-u128.rs', > + 'tests/codegen/fn-impl-trait-self.rs', > 'tests/codegen/force-unwind-tables.rs', > + 'tests/codegen/inherit_overflow.rs', > + 'tests/codegen/inherit_overflow.rs', > + 'tests/codegen/inline-function-args-debug-info.rs', > 'tests/codegen/intrinsic-no-unnamed-attr.rs', > + 'tests/codegen/intrinsics/mask.rs', > + 'tests/codegen/intrinsics/transmute-niched.rs', > 'tests/codegen/issues/issue-103840.rs', > 'tests/codegen/issues/issue-47278.rs', > + 'tests/codegen/issues/issue-73258.rs', > 'tests/codegen/issues/issue-73827-bounds-check-index-in-subexpr.rs', > + 'tests/codegen/issues/issue-75546.rs', > + 'tests/codegen/issues/issue-77812.rs', > + 'tests/codegen/issues/issue-98156-const-arg-temp-lifetime.rs', > 'tests/codegen/lifetime_start_end.rs', > + 'tests/codegen/llvm-ident.rs', > 'tests/codegen/local-generics-in-exe-internalized.rs', > + 'tests/codegen/mainsubprogram.rs', > 'tests/codegen/match-unoptimized.rs', > + 'tests/codegen/move-operands.rs', > 'tests/codegen/noalias-rwlockreadguard.rs', > 'tests/codegen/non-terminate/nonempty-infinite-loop.rs', > 'tests/codegen/noreturn-uninhabited.rs', > 'tests/codegen/repr-transparent-aggregates-3.rs', > + 'tests/codegen/repr/transparent-mips64.rs' > 'tests/codegen/sse42-implies-crc32.rs', > 'tests/codegen/thread-local.rs', > 'tests/codegen/uninit-consts.rs', > + 'tests/mir-opt/', > 'tests/pretty/raw-str-nonexpr.rs', > 'tests/run-make', > + 'tests/run-make-fulldeps', > 'tests/run-make/cdylib-fewer-symbols/foo.rs', > 'tests/run-make/doctests-keep-binaries/t.rs', > - 'tests/run-make-fulldeps', > 'tests/run-make/issue-22131/foo.rs', > 'tests/run-make/issue-36710/Makefile', > 'tests/run-make/issue-47551', > @@ -119,6 +152,26 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > 'tests/run-make/sysroot-crates-are-unstable', > 'tests/run-make/target-specs', > 'tests/rustdoc', > + 'tests/rustdoc-js-std', > + 'tests/rustdoc-json', > + 'tests/rustdoc-ui/cfg-test.rs', > + 'tests/rustdoc-ui/check-cfg-test.rs', > + 'tests/rustdoc-ui/display-output.rs', > + 'tests/rustdoc-ui/doc-comment-multi-line-attr.rs', > + 'tests/rustdoc-ui/doc-comment-multi-line-cfg-attr.rs', > + 'tests/rustdoc-ui/doc-test-doctest-feature.rs', > + 'tests/rustdoc-ui/doc-test-rustdoc-feature.rs', > + 'tests/rustdoc-ui/doctest-multiline-crate-attribute.rs', > + 'tests/rustdoc-ui/doctest-output.rs', > + 'tests/rustdoc-ui/failed-doctest-compile-fail.rs', > + 'tests/rustdoc-ui/issue-80992.rs', > + 'tests/rustdoc-ui/issue-91134.rs', > + 'tests/rustdoc-ui/no-run-flag.rs', > + 'tests/rustdoc-ui/nocapture-fail.rs', > + 'tests/rustdoc-ui/nocapture.rs', > + 'tests/rustdoc-ui/run-directory.rs', > + 'tests/rustdoc-ui/test-no_std.rs', > + 'tests/rustdoc-ui/test-type.rs', > 'tests/rustdoc/async-move-doctest.rs', > 'tests/rustdoc/async-trait.rs', > 'tests/rustdoc/auto-traits.rs', > @@ -133,9 +186,9 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > 'tests/rustdoc/edition-doctest.rs', > 'tests/rustdoc/edition-flag.rs', > 'tests/rustdoc/elided-lifetime.rs', > - 'tests/rustdoc/external-macro-src.rs', > 'tests/rustdoc/extern-html-root-url.rs', > 'tests/rustdoc/extern-impl-trait.rs', > + 'tests/rustdoc/external-macro-src.rs', > 'tests/rustdoc/hide-unstable-trait.rs', > 'tests/rustdoc/inline_cross/add-docs.rs', > 'tests/rustdoc/inline_cross/default-trait-method.rs', > @@ -184,12 +237,10 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > 'tests/rustdoc/issue-75588.rs', > 'tests/rustdoc/issue-85454.rs', > 'tests/rustdoc/issue-86620.rs', > - 'tests/rustdoc-json', > - 'tests/rustdoc-js-std', > 'tests/rustdoc/macro_pub_in_module.rs', > 'tests/rustdoc/masked.rs', > - 'tests/rustdoc/normalize-assoc-item.rs', > 'tests/rustdoc/no-stack-overflow-25295.rs', > + 'tests/rustdoc/normalize-assoc-item.rs', > 'tests/rustdoc/primitive-reexport.rs', > 'tests/rustdoc/process-termination.rs', > 'tests/rustdoc/pub-extern-crate.rs', > @@ -204,35 +255,8 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > 'tests/rustdoc/test_option_check/test.rs', > 'tests/rustdoc/trait-alias-mention.rs', > 'tests/rustdoc/trait-visibility.rs', > - 'tests/rustdoc-ui/cfg-test.rs', > - 'tests/rustdoc-ui/check-cfg-test.rs', > - 'tests/rustdoc-ui/display-output.rs', > - 'tests/rustdoc-ui/doc-comment-multi-line-attr.rs', > - 'tests/rustdoc-ui/doc-comment-multi-line-cfg-attr.rs', > - 'tests/rustdoc-ui/doc-test-doctest-feature.rs', > - 'tests/rustdoc-ui/doctest-multiline-crate-attribute.rs', > - 'tests/rustdoc-ui/doctest-output.rs', > - 'tests/rustdoc-ui/doc-test-rustdoc-feature.rs', > - 'tests/rustdoc-ui/failed-doctest-compile-fail.rs', > - 'tests/rustdoc-ui/issue-80992.rs', > - 'tests/rustdoc-ui/issue-91134.rs', > - 'tests/rustdoc-ui/nocapture-fail.rs', > - 'tests/rustdoc-ui/nocapture.rs', > - 'tests/rustdoc-ui/no-run-flag.rs', > - 'tests/rustdoc-ui/run-directory.rs', > - 'tests/rustdoc-ui/test-no_std.rs', > - 'tests/rustdoc-ui/test-type.rs', > 'tests/rustdoc/unit-return.rs', > - 'tests/ui/abi/stack-probes-lto.rs', > - 'tests/ui/abi/stack-probes.rs', > - 'tests/ui/array-slice-vec/subslice-patterns-const-eval-match.rs', > - 'tests/ui/asm/x86_64/sym.rs', > - 'tests/ui/associated-type-bounds/fn-apit.rs', > - 'tests/ui/associated-type-bounds/fn-dyn-apit.rs', > - 'tests/ui/associated-type-bounds/fn-wrap-apit.rs', > - 'tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs', > - 'tests/ui/drop/dynamic-drop.rs', > - 'tests/ui/empty_global_asm.rs', > + 'tests/ui-fulldeps/', > 'tests/ui-fulldeps/deriving-encodable-decodable-box.rs', > 'tests/ui-fulldeps/deriving-encodable-decodable-cell-refcell.rs', > 'tests/ui-fulldeps/deriving-global.rs', > @@ -255,9 +279,20 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): > 'tests/ui-fulldeps/regions-mock-tcx.rs', > 'tests/ui-fulldeps/rustc_encodable_hygiene.rs', > 'tests/ui-fulldeps/session-diagnostic/enforce_slug_naming.rs', > + 'tests/ui/abi/stack-probes-lto.rs', > + 'tests/ui/abi/stack-probes.rs', > + 'tests/ui/array-slice-vec/subslice-patterns-const-eval-match.rs', > + 'tests/ui/asm/x86_64/sym.rs', > + 'tests/ui/associated-type-bounds/fn-apit.rs', > + 'tests/ui/associated-type-bounds/fn-dyn-apit.rs', > + 'tests/ui/associated-type-bounds/fn-wrap-apit.rs', > + 'tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs', > + 'tests/ui/drop/dynamic-drop.rs', > + 'tests/ui/empty_global_asm.rs', > 'tests/ui/functions-closures/fn-help-with-err.rs', > 'tests/ui/linkage-attr/issue-10755.rs', > 'tests/ui/macros/restricted-shadowing-legacy.rs', > + 'tests/ui/numbers-arithmetic/u128.rs', > 'tests/ui/process/nofile-limit.rs', > 'tests/ui/process/process-panic-after-fork.rs', > 'tests/ui/process/process-sigpipe.rs', > diff --git a/meta/recipes-devtools/rust/files/target_cfg.patch b/meta/recipes-devtools/rust/files/target_cfg.patch > new file mode 100644 > index 0000000000..b6a80e004f > --- /dev/null > +++ b/meta/recipes-devtools/rust/files/target_cfg.patch > @@ -0,0 +1,144 @@ > +rust: Fix various failures for Rust Oe-selftest > + > +* Ensure cargo executable is fetched > + > +* Fix the incorrect target on stage1 ui-fulldeps tests > + > +* Enable RUSTC_BOOTSTRAP on panic=abort mir-opt test > + > +* Detect user-specified custom targets in compiletest > + > +Upstream-Status: Backport [https://github.com/rust-lang/rust/pull/119619/commits] > + > +Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com> > +--- > +diff --git a/src/bootstrap/bootstrap.py b/src/bootstrap/bootstrap.py > +--- a/src/bootstrap/bootstrap.py > ++++ b/src/bootstrap/bootstrap.py > +@@ -954,9 +954,11 @@ > + if deny_warnings: > + env["RUSTFLAGS"] += " -Dwarnings" > + > +- env["PATH"] = os.path.join(self.bin_root(), "bin") + \ > +- os.pathsep + env["PATH"] > +- if not os.path.isfile(self.cargo()): > ++ cargo_bin_path = os.path.join(self.bin_root(), "bin", "cargo") > ++ if not os.path.isfile(cargo_bin_path): > ++ cargo_bin_path = os.getenv("RUST_TARGET_PATH") + "rust-snapshot/bin/cargo" > ++ env["PATH"] = os.path.dirname(cargo_bin_path) + os.pathsep + env["PATH"] > ++ else: > + raise Exception("no cargo executable found at `{}`".format( > + self.cargo())) > + args = [self.cargo(), "build", "--manifest-path", > +diff --git a/src/bootstrap/test.rs b/src/bootstrap/test.rs > +--- a/src/bootstrap/test.rs > ++++ b/src/bootstrap/test.rs > +@@ -1489,8 +1489,12 @@ > + // NOTE: Only stage 1 is special cased because we need the rustc_private artifacts to match the > + // running compiler in stage 2 when plugins run. > + let stage_id = if suite == "ui-fulldeps" && compiler.stage == 1 { > +- compiler = builder.compiler(compiler.stage - 1, target); > +- format!("stage{}-{}", compiler.stage + 1, target) > ++ // At stage 0 (stage - 1) we are using the beta compiler. Using `self.target` can lead finding > ++ // an incorrect compiler path on cross-targets, as the stage 0 beta compiler is always equal > ++ // to `build.build` in the configuration. > ++ let build = builder.build.build; > ++ compiler = builder.compiler(compiler.stage - 1, build); > ++ format!("stage{}-{}", compiler.stage + 1, build) > + } else { > + format!("stage{}-{}", compiler.stage, target) > + }; > +diff --git a/src/bootstrap/synthetic_targets.rs b/ src/bootstrap/synthetic_targets.rs > +index d2c65b740da..45baf56f46b 100644 > +--- a/src/bootstrap/synthetic_targets.rs > ++++ b/src/bootstrap/synthetic_targets.rs > +@@ -59,6 +59,7 @@ fn create_synthetic_target( > + let mut cmd = Command::new(builder.rustc(compiler)); > + cmd.arg("--target").arg(base.rustc_target_arg()); > + cmd.args(["-Zunstable-options", "--print", "target-spec-json"]); > ++ cmd.env("RUSTC_BOOTSTRAP", "1"); > + cmd.stdout(Stdio::piped()); > + > + let output = cmd.spawn().unwrap().wait_with_output().unwrap(); > +diff --git a/src/tools/compiletest/src/common.rs b/src/tools/compiletest/src/common.rs > +index e85f6319936..c45c0b3c652 100644 > +--- a/src/tools/compiletest/src/common.rs > ++++ b/src/tools/compiletest/src/common.rs > +@@ -479,6 +479,7 @@ fn new(config: &Config) -> TargetCfgs { > + let mut targets: HashMap<String, TargetCfg> = serde_json::from_str(&rustc_output( > + config, > + &["--print=all-target-specs-json", "-Zunstable-options"], > ++ Default::default(), > + )) > + .unwrap(); > + > +@@ -491,16 +492,33 @@ fn new(config: &Config) -> TargetCfgs { > + let mut all_families = HashSet::new(); > + let mut all_pointer_widths = HashSet::new(); > + > +- // Handle custom target specs, which are not included in `--print=all-target-specs-json`. > +- if config.target.ends_with(".json") { > +- targets.insert( > +- config.target.clone(), > +- serde_json::from_str(&rustc_output( > +- config, > +- &["--print=target-spec-json", "-Zunstable-options", "--target", &config.target], > +- )) > +- .unwrap(), > +- ); > ++ // If current target is not included in the `--print=all-target-specs-json` output, > ++ // we check whether it is a custom target from the user or a synthetic target from bootstrap. > ++ if !targets.contains_key(&config.target) { > ++ let mut envs: HashMap<String, String> = HashMap::new(); > ++ > ++ if let Ok(t) = std::env::var("RUST_TARGET_PATH") { > ++ envs.insert("RUST_TARGET_PATH".into(), t); > ++ } > ++ > ++ // This returns false only when the target is neither a synthetic target > ++ // nor a custom target from the user, indicating it is most likely invalid. > ++ if config.target.ends_with(".json") || !envs.is_empty() { > ++ targets.insert( > ++ config.target.clone(), > ++ serde_json::from_str(&rustc_output( > ++ config, > ++ &[ > ++ "--print=target-spec-json", > ++ "-Zunstable-options", > ++ "--target", > ++ &config.target, > ++ ], > ++ envs, > ++ )) > ++ .unwrap(), > ++ ); > ++ } > + } > + > + for (target, cfg) in targets.iter() { > +@@ -545,7 +563,9 @@ fn get_current_target_config( > + // code below extracts them from `--print=cfg`: make sure to only override fields that can > + // actually be changed with `-C` flags. > + for config in > +- rustc_output(config, &["--print=cfg", "--target", &config.target]).trim().lines() > ++ rustc_output(config, &["--print=cfg", "--target", &config.target], Default::default()) > ++ .trim() > ++ .lines() > + { > + let (name, value) = config > + .split_once("=\"") > +@@ -624,11 +644,12 @@ pub enum Endian { > + Big, > + } > + > +-fn rustc_output(config: &Config, args: &[&str]) -> String { > ++fn rustc_output(config: &Config, args: &[&str], envs: HashMap<String, String>) -> String { > + let mut command = Command::new(&config.rustc_path); > + add_dylib_path(&mut command, iter::once(&config.compile_lib_path)); > + command.args(&config.target_rustcflags).args(args); > + command.env("RUSTC_BOOTSTRAP", "1"); > ++ command.envs(envs); > + > + let output = match command.output() { > + Ok(output) => output, > + > diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc > index 83a0dbc15f..d9a370ac2f 100644 > --- a/meta/recipes-devtools/rust/rust-source.inc > +++ b/meta/recipes-devtools/rust/rust-source.inc > @@ -10,6 +10,7 @@ SRC_URI +="https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n > file://0004-musl-Define-O_LARGEFILE-for-riscv32.patch;patchdir=${RUSTSRC} > \ > file://0005-musl-Define-SOCK_SEQPACKET-in-common-place.patch;patchdir=${RUSTSRC} > \ > file://0001-Revert-Map-source-absolute-paths-to-OUT_DIR-as-relat.patch;patchdir=${RUSTSRC} > \ + file://target_cfg.patch;patchdir=${RUSTSRC} \ " > SRC_URI[rust.sha256sum] = "b98c09d968529212fb29eec7d6d3e9bdaa869810679b7fb86a1ca69469d75f5e" >
On 19-01-2024 01:38, Randy MacLeod wrote: > On 2024-01-18 5:57 a.m., Yash.Shinde@windriver.com wrote: >> From: Yash Shinde<Yash.Shinde@windriver.com> >> >> Enable rust oe-selftest for rust 1.74.1 version and >> add target_cfg.patch to handle target configurations >> for custom targets. > > Hi Yashe, > > Yay, getting the rust test suite working again great news! > > We talked about this in today's YP bug review and decided NOT to merge > this > until after the M2 build is taken care of over the next few days. > > That gives me a chance to review the commit and ask some questions! ;-) > > First, your commit log should be expanded to explain what has changed > at a high level > to someone who is just reading "git log". > > What you have is a good start but I'd like more info on what changed in: > meta/lib/oeqa/selftest/cases/rust.py > > There are lots of tests added to the exclude list and some seem to > have just moved. > > Please explain why you did this and if there is any way to avoid the > "noise" of moved tests in future updates. Could you also report on the > number of newly disabled tests and try to summarize in the commit log > what these tests are doing and > > Ah, I see what you have done: 1. sort the exclude list 2. add the > newly excluded tests. Please do that in separate commits so that it's > easier to understand. This will reduce the size of the "Enable rust > oe-selftest" diff for meta/lib/oeqa/selftest/cases/rust.py from: | 103 > ++++++++----- to: | 37 +++- > > and let people easily see that most of the newly excluded tests are in > tests/codegen > > That reminds me, do we really need to list directories in that exclude > list or only list .rs files? > > If I save the list to /tmp/jj, you can see that we could reduce the > list by 34 lines: > > ❯ rg "\.rs" /tmp/jj | wc -l 203 ❯ rg -v "\.rs" /tmp/jj | wc -l 34 > > If we can do that, again do it as a separate commit since it should > not affect the outcome of the test. > > > Also this backport link: > https://github.com/rust-lang/rust/pull/119619/commits > is actually 3 separate commits so it might be better to apply those > individually > so that we can know when they can be dropped by running: > $ git tag --contains <commit-id> on each commit > > Also in the patches, please add you Signed-Off-By line but keep the > upstream author's S-O-B line. > > I will do patch changes to split into separate commits for each issue fix, and with oe-selftest changes. Exclude list is sorted and updated w/ module names, and now the module names and test list reduced to 114 lines from 201 lines. I will send a v2 with updating above changes. Regards, Yash > > For anyone interested we did talk about the number of Rust tests that > we are excluding and of course our goal is to work with the upstream > Rust community to reduce the number of excluded tests over the coming > months but for now, it's just nice to see the test working again! > > If you update these things convince yourself that the changes are > functionally identical, I don't think there's any need to re-do all > the testing that you have done. > > Thanks Yash, > > ../Randy > > > -- > # Randy MacLeod > # Wind River Linux
diff --git a/meta/lib/oeqa/selftest/cases/rust.py b/meta/lib/oeqa/selftest/cases/rust.py index 6dbc517006..216c73e5d1 100644 --- a/meta/lib/oeqa/selftest/cases/rust.py +++ b/meta/lib/oeqa/selftest/cases/rust.py @@ -40,7 +40,7 @@ def parse_results(filename): class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): def test_rust(self, *args, **kwargs): # Disable Rust Oe-selftest - self.skipTest("The Rust Oe-selftest is disabled.") + #self.skipTest("The Rust Oe-selftest is disabled.") # build remote-test-server before image build recipe = "rust" @@ -73,38 +73,71 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): 'src/librustdoc', 'src/rustdoc-json-types', 'src/tools/compiletest/src/common.rs', + 'src/tools/jsondoclint', 'src/tools/lint-docs', + 'src/tools/replace-version-placeholder', 'src/tools/rust-analyzer', 'src/tools/rustdoc-themes', 'src/tools/tidy', 'tests/assembly/asm/aarch64-outline-atomics.rs', + 'tests/codegen/abi-efiapi.rs', + 'tests/codegen/abi-efiapi.rs', 'tests/codegen/abi-main-signature-32bit-c-int.rs', 'tests/codegen/abi-repr-ext.rs', + 'tests/codegen/abi-sysv64.rs', 'tests/codegen/abi-x86-interrupt.rs', + 'tests/codegen/align-byval.rs', + 'tests/codegen/align-fn.rs', + 'tests/codegen/asm-powerpc-clobbers.rs', + 'tests/codegen/async-fn-debug-awaitee-field.rs', + 'tests/codegen/binary-search-index-no-bound-check.rs', 'tests/codegen/branch-protection.rs', + 'tests/codegen/call-metadata.rs', 'tests/codegen/catch-unwind.rs', 'tests/codegen/cf-protection.rs', + 'tests/codegen/debug-column.rs', + 'tests/codegen/debug-limited.rs', + 'tests/codegen/debuginfo-generic-closure-env-names.rs', + 'tests/codegen/drop.rs', + 'tests/codegen/dst-vtable-align-nonzero.rs', + 'tests/codegen/enable-lto-unit-splitting.rs', 'tests/codegen/enum-bounds-check-derived-idx.rs', + 'tests/codegen/enum/enum-u128.rs', + 'tests/codegen/fn-impl-trait-self.rs', 'tests/codegen/force-unwind-tables.rs', + 'tests/codegen/inherit_overflow.rs', + 'tests/codegen/inherit_overflow.rs', + 'tests/codegen/inline-function-args-debug-info.rs', 'tests/codegen/intrinsic-no-unnamed-attr.rs', + 'tests/codegen/intrinsics/mask.rs', + 'tests/codegen/intrinsics/transmute-niched.rs', 'tests/codegen/issues/issue-103840.rs', 'tests/codegen/issues/issue-47278.rs', + 'tests/codegen/issues/issue-73258.rs', 'tests/codegen/issues/issue-73827-bounds-check-index-in-subexpr.rs', + 'tests/codegen/issues/issue-75546.rs', + 'tests/codegen/issues/issue-77812.rs', + 'tests/codegen/issues/issue-98156-const-arg-temp-lifetime.rs', 'tests/codegen/lifetime_start_end.rs', + 'tests/codegen/llvm-ident.rs', 'tests/codegen/local-generics-in-exe-internalized.rs', + 'tests/codegen/mainsubprogram.rs', 'tests/codegen/match-unoptimized.rs', + 'tests/codegen/move-operands.rs', 'tests/codegen/noalias-rwlockreadguard.rs', 'tests/codegen/non-terminate/nonempty-infinite-loop.rs', 'tests/codegen/noreturn-uninhabited.rs', 'tests/codegen/repr-transparent-aggregates-3.rs', + 'tests/codegen/repr/transparent-mips64.rs' 'tests/codegen/sse42-implies-crc32.rs', 'tests/codegen/thread-local.rs', 'tests/codegen/uninit-consts.rs', + 'tests/mir-opt/', 'tests/pretty/raw-str-nonexpr.rs', 'tests/run-make', + 'tests/run-make-fulldeps', 'tests/run-make/cdylib-fewer-symbols/foo.rs', 'tests/run-make/doctests-keep-binaries/t.rs', - 'tests/run-make-fulldeps', 'tests/run-make/issue-22131/foo.rs', 'tests/run-make/issue-36710/Makefile', 'tests/run-make/issue-47551', @@ -119,6 +152,26 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): 'tests/run-make/sysroot-crates-are-unstable', 'tests/run-make/target-specs', 'tests/rustdoc', + 'tests/rustdoc-js-std', + 'tests/rustdoc-json', + 'tests/rustdoc-ui/cfg-test.rs', + 'tests/rustdoc-ui/check-cfg-test.rs', + 'tests/rustdoc-ui/display-output.rs', + 'tests/rustdoc-ui/doc-comment-multi-line-attr.rs', + 'tests/rustdoc-ui/doc-comment-multi-line-cfg-attr.rs', + 'tests/rustdoc-ui/doc-test-doctest-feature.rs', + 'tests/rustdoc-ui/doc-test-rustdoc-feature.rs', + 'tests/rustdoc-ui/doctest-multiline-crate-attribute.rs', + 'tests/rustdoc-ui/doctest-output.rs', + 'tests/rustdoc-ui/failed-doctest-compile-fail.rs', + 'tests/rustdoc-ui/issue-80992.rs', + 'tests/rustdoc-ui/issue-91134.rs', + 'tests/rustdoc-ui/no-run-flag.rs', + 'tests/rustdoc-ui/nocapture-fail.rs', + 'tests/rustdoc-ui/nocapture.rs', + 'tests/rustdoc-ui/run-directory.rs', + 'tests/rustdoc-ui/test-no_std.rs', + 'tests/rustdoc-ui/test-type.rs', 'tests/rustdoc/async-move-doctest.rs', 'tests/rustdoc/async-trait.rs', 'tests/rustdoc/auto-traits.rs', @@ -133,9 +186,9 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): 'tests/rustdoc/edition-doctest.rs', 'tests/rustdoc/edition-flag.rs', 'tests/rustdoc/elided-lifetime.rs', - 'tests/rustdoc/external-macro-src.rs', 'tests/rustdoc/extern-html-root-url.rs', 'tests/rustdoc/extern-impl-trait.rs', + 'tests/rustdoc/external-macro-src.rs', 'tests/rustdoc/hide-unstable-trait.rs', 'tests/rustdoc/inline_cross/add-docs.rs', 'tests/rustdoc/inline_cross/default-trait-method.rs', @@ -184,12 +237,10 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): 'tests/rustdoc/issue-75588.rs', 'tests/rustdoc/issue-85454.rs', 'tests/rustdoc/issue-86620.rs', - 'tests/rustdoc-json', - 'tests/rustdoc-js-std', 'tests/rustdoc/macro_pub_in_module.rs', 'tests/rustdoc/masked.rs', - 'tests/rustdoc/normalize-assoc-item.rs', 'tests/rustdoc/no-stack-overflow-25295.rs', + 'tests/rustdoc/normalize-assoc-item.rs', 'tests/rustdoc/primitive-reexport.rs', 'tests/rustdoc/process-termination.rs', 'tests/rustdoc/pub-extern-crate.rs', @@ -204,35 +255,8 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): 'tests/rustdoc/test_option_check/test.rs', 'tests/rustdoc/trait-alias-mention.rs', 'tests/rustdoc/trait-visibility.rs', - 'tests/rustdoc-ui/cfg-test.rs', - 'tests/rustdoc-ui/check-cfg-test.rs', - 'tests/rustdoc-ui/display-output.rs', - 'tests/rustdoc-ui/doc-comment-multi-line-attr.rs', - 'tests/rustdoc-ui/doc-comment-multi-line-cfg-attr.rs', - 'tests/rustdoc-ui/doc-test-doctest-feature.rs', - 'tests/rustdoc-ui/doctest-multiline-crate-attribute.rs', - 'tests/rustdoc-ui/doctest-output.rs', - 'tests/rustdoc-ui/doc-test-rustdoc-feature.rs', - 'tests/rustdoc-ui/failed-doctest-compile-fail.rs', - 'tests/rustdoc-ui/issue-80992.rs', - 'tests/rustdoc-ui/issue-91134.rs', - 'tests/rustdoc-ui/nocapture-fail.rs', - 'tests/rustdoc-ui/nocapture.rs', - 'tests/rustdoc-ui/no-run-flag.rs', - 'tests/rustdoc-ui/run-directory.rs', - 'tests/rustdoc-ui/test-no_std.rs', - 'tests/rustdoc-ui/test-type.rs', 'tests/rustdoc/unit-return.rs', - 'tests/ui/abi/stack-probes-lto.rs', - 'tests/ui/abi/stack-probes.rs', - 'tests/ui/array-slice-vec/subslice-patterns-const-eval-match.rs', - 'tests/ui/asm/x86_64/sym.rs', - 'tests/ui/associated-type-bounds/fn-apit.rs', - 'tests/ui/associated-type-bounds/fn-dyn-apit.rs', - 'tests/ui/associated-type-bounds/fn-wrap-apit.rs', - 'tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs', - 'tests/ui/drop/dynamic-drop.rs', - 'tests/ui/empty_global_asm.rs', + 'tests/ui-fulldeps/', 'tests/ui-fulldeps/deriving-encodable-decodable-box.rs', 'tests/ui-fulldeps/deriving-encodable-decodable-cell-refcell.rs', 'tests/ui-fulldeps/deriving-global.rs', @@ -255,9 +279,20 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): 'tests/ui-fulldeps/regions-mock-tcx.rs', 'tests/ui-fulldeps/rustc_encodable_hygiene.rs', 'tests/ui-fulldeps/session-diagnostic/enforce_slug_naming.rs', + 'tests/ui/abi/stack-probes-lto.rs', + 'tests/ui/abi/stack-probes.rs', + 'tests/ui/array-slice-vec/subslice-patterns-const-eval-match.rs', + 'tests/ui/asm/x86_64/sym.rs', + 'tests/ui/associated-type-bounds/fn-apit.rs', + 'tests/ui/associated-type-bounds/fn-dyn-apit.rs', + 'tests/ui/associated-type-bounds/fn-wrap-apit.rs', + 'tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs', + 'tests/ui/drop/dynamic-drop.rs', + 'tests/ui/empty_global_asm.rs', 'tests/ui/functions-closures/fn-help-with-err.rs', 'tests/ui/linkage-attr/issue-10755.rs', 'tests/ui/macros/restricted-shadowing-legacy.rs', + 'tests/ui/numbers-arithmetic/u128.rs', 'tests/ui/process/nofile-limit.rs', 'tests/ui/process/process-panic-after-fork.rs', 'tests/ui/process/process-sigpipe.rs', diff --git a/meta/recipes-devtools/rust/files/target_cfg.patch b/meta/recipes-devtools/rust/files/target_cfg.patch new file mode 100644 index 0000000000..b6a80e004f --- /dev/null +++ b/meta/recipes-devtools/rust/files/target_cfg.patch @@ -0,0 +1,144 @@ +rust: Fix various failures for Rust Oe-selftest + +* Ensure cargo executable is fetched + +* Fix the incorrect target on stage1 ui-fulldeps tests + +* Enable RUSTC_BOOTSTRAP on panic=abort mir-opt test + +* Detect user-specified custom targets in compiletest + +Upstream-Status: Backport [https://github.com/rust-lang/rust/pull/119619/commits] + +Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com> +--- +diff --git a/src/bootstrap/bootstrap.py b/src/bootstrap/bootstrap.py +--- a/src/bootstrap/bootstrap.py ++++ b/src/bootstrap/bootstrap.py +@@ -954,9 +954,11 @@ + if deny_warnings: + env["RUSTFLAGS"] += " -Dwarnings" + +- env["PATH"] = os.path.join(self.bin_root(), "bin") + \ +- os.pathsep + env["PATH"] +- if not os.path.isfile(self.cargo()): ++ cargo_bin_path = os.path.join(self.bin_root(), "bin", "cargo") ++ if not os.path.isfile(cargo_bin_path): ++ cargo_bin_path = os.getenv("RUST_TARGET_PATH") + "rust-snapshot/bin/cargo" ++ env["PATH"] = os.path.dirname(cargo_bin_path) + os.pathsep + env["PATH"] ++ else: + raise Exception("no cargo executable found at `{}`".format( + self.cargo())) + args = [self.cargo(), "build", "--manifest-path", +diff --git a/src/bootstrap/test.rs b/src/bootstrap/test.rs +--- a/src/bootstrap/test.rs ++++ b/src/bootstrap/test.rs +@@ -1489,8 +1489,12 @@ + // NOTE: Only stage 1 is special cased because we need the rustc_private artifacts to match the + // running compiler in stage 2 when plugins run. + let stage_id = if suite == "ui-fulldeps" && compiler.stage == 1 { +- compiler = builder.compiler(compiler.stage - 1, target); +- format!("stage{}-{}", compiler.stage + 1, target) ++ // At stage 0 (stage - 1) we are using the beta compiler. Using `self.target` can lead finding ++ // an incorrect compiler path on cross-targets, as the stage 0 beta compiler is always equal ++ // to `build.build` in the configuration. ++ let build = builder.build.build; ++ compiler = builder.compiler(compiler.stage - 1, build); ++ format!("stage{}-{}", compiler.stage + 1, build) + } else { + format!("stage{}-{}", compiler.stage, target) + }; +diff --git a/src/bootstrap/synthetic_targets.rs b/ src/bootstrap/synthetic_targets.rs +index d2c65b740da..45baf56f46b 100644 +--- a/src/bootstrap/synthetic_targets.rs ++++ b/src/bootstrap/synthetic_targets.rs +@@ -59,6 +59,7 @@ fn create_synthetic_target( + let mut cmd = Command::new(builder.rustc(compiler)); + cmd.arg("--target").arg(base.rustc_target_arg()); + cmd.args(["-Zunstable-options", "--print", "target-spec-json"]); ++ cmd.env("RUSTC_BOOTSTRAP", "1"); + cmd.stdout(Stdio::piped()); + + let output = cmd.spawn().unwrap().wait_with_output().unwrap(); +diff --git a/src/tools/compiletest/src/common.rs b/src/tools/compiletest/src/common.rs +index e85f6319936..c45c0b3c652 100644 +--- a/src/tools/compiletest/src/common.rs ++++ b/src/tools/compiletest/src/common.rs +@@ -479,6 +479,7 @@ fn new(config: &Config) -> TargetCfgs { + let mut targets: HashMap<String, TargetCfg> = serde_json::from_str(&rustc_output( + config, + &["--print=all-target-specs-json", "-Zunstable-options"], ++ Default::default(), + )) + .unwrap(); + +@@ -491,16 +492,33 @@ fn new(config: &Config) -> TargetCfgs { + let mut all_families = HashSet::new(); + let mut all_pointer_widths = HashSet::new(); + +- // Handle custom target specs, which are not included in `--print=all-target-specs-json`. +- if config.target.ends_with(".json") { +- targets.insert( +- config.target.clone(), +- serde_json::from_str(&rustc_output( +- config, +- &["--print=target-spec-json", "-Zunstable-options", "--target", &config.target], +- )) +- .unwrap(), +- ); ++ // If current target is not included in the `--print=all-target-specs-json` output, ++ // we check whether it is a custom target from the user or a synthetic target from bootstrap. ++ if !targets.contains_key(&config.target) { ++ let mut envs: HashMap<String, String> = HashMap::new(); ++ ++ if let Ok(t) = std::env::var("RUST_TARGET_PATH") { ++ envs.insert("RUST_TARGET_PATH".into(), t); ++ } ++ ++ // This returns false only when the target is neither a synthetic target ++ // nor a custom target from the user, indicating it is most likely invalid. ++ if config.target.ends_with(".json") || !envs.is_empty() { ++ targets.insert( ++ config.target.clone(), ++ serde_json::from_str(&rustc_output( ++ config, ++ &[ ++ "--print=target-spec-json", ++ "-Zunstable-options", ++ "--target", ++ &config.target, ++ ], ++ envs, ++ )) ++ .unwrap(), ++ ); ++ } + } + + for (target, cfg) in targets.iter() { +@@ -545,7 +563,9 @@ fn get_current_target_config( + // code below extracts them from `--print=cfg`: make sure to only override fields that can + // actually be changed with `-C` flags. + for config in +- rustc_output(config, &["--print=cfg", "--target", &config.target]).trim().lines() ++ rustc_output(config, &["--print=cfg", "--target", &config.target], Default::default()) ++ .trim() ++ .lines() + { + let (name, value) = config + .split_once("=\"") +@@ -624,11 +644,12 @@ pub enum Endian { + Big, + } + +-fn rustc_output(config: &Config, args: &[&str]) -> String { ++fn rustc_output(config: &Config, args: &[&str], envs: HashMap<String, String>) -> String { + let mut command = Command::new(&config.rustc_path); + add_dylib_path(&mut command, iter::once(&config.compile_lib_path)); + command.args(&config.target_rustcflags).args(args); + command.env("RUSTC_BOOTSTRAP", "1"); ++ command.envs(envs); + + let output = match command.output() { + Ok(output) => output, + diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc index 83a0dbc15f..d9a370ac2f 100644 --- a/meta/recipes-devtools/rust/rust-source.inc +++ b/meta/recipes-devtools/rust/rust-source.inc @@ -10,6 +10,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n file://0004-musl-Define-O_LARGEFILE-for-riscv32.patch;patchdir=${RUSTSRC} \ file://0005-musl-Define-SOCK_SEQPACKET-in-common-place.patch;patchdir=${RUSTSRC} \ file://0001-Revert-Map-source-absolute-paths-to-OUT_DIR-as-relat.patch;patchdir=${RUSTSRC} \ + file://target_cfg.patch;patchdir=${RUSTSRC} \ " SRC_URI[rust.sha256sum] = "b98c09d968529212fb29eec7d6d3e9bdaa869810679b7fb86a1ca69469d75f5e"