[RFC,0/6] (e)SDK workflow directly in a Yocto build

There's been a recent discussion about how we can make the Yocto SDK
experience better [1]. One of the ideas was to eliminate the SDK
as a separate artefact altogether and simply provide everything
that the SDK and eSDKs do directly in a yocto build. This does not
mean that people have to 'learn Yocto', but rather that the integrators
should provide a well-functioning sstate cache infrastructure (same as
with minimal eSDK, really), and a few wrapper scripts for setting up the build
and the SDK environment that run layer setup and bitbake behind the scenes.

[1] https://lists.openembedded.org/g/openembedded-architecture/topic/thoughts_on_the_esdk/90990557

So without further ado, here's how you get a 'SDK' with this set of patches:

1. Set up all the needed layers and a yocto build directory.

2. Run:
$ bitbake meta-ide-support
$ bitbake -c populate_sysroot gtk+3
(or any other target or native item that the application developer would need)
$ bitbake populate-sysroots

3. Set up the SDK environment:
. tmp/deploy/images/qemux86-64/environment-setup-core2-64-poky-linux
(adjust accordingly)

Et voila! The Unix environment is now set up to use the cross-toolchain from
Yocto, exactly as in the SDK. And devtool/bitbake are available to extend it,
exactly as in the eSDK.

Theare are numerous benefits here: no need to produce, test, distribute and maintain
separate SDK artifacts. No two separate environments for the yocto build and the SDK.
Less code paths where things can go wrong. Less awkward, gigantic tarballs. Less
SDK update headaches: 'updating the SDK' simply means updating the yocto layers with
git fetch or layer management tooling. Built-in SDK extensibility: just run bitbake
again to add more things to the sysroot, or add layers if even more things are required.

How is this tested?

Exactly same as the regular SDK:
$ bitbake -c testsdk meta-ide-support

This runs the same toolchain tests from meta/lib/oeqa/sdk/cases as the regular
sdk testing does.

One known issue is that the libepoxy test is failing: libepoxy is built with meson,
and meson requires the build configuration to be provided with native/cross files and not
in the unix environment. This is addressed in the 'classic SDK' through a wrapper for the binary,
and something similar will be added to meta-ide-support as well.


Alexander Kanavin (6):
  toolchain-scripts.bbclass: adjust toolchain_create_tree_env_script to
    better replicate (e)SDK
  meta-ide-support: adjust to provide (e)SDK experience directly in a
    yocto build
  oeqa/sdk: add a test class for running SDK tests directly in a Yocto
    build
  selftest/meta_ide: add a test for running SDK tests
  oeqa/sdk: allow epoxy/galculator tests to run in esdk and direct yocto
    builds
  oeqa/sdk: drop the nativesdk-python 2.x test

 meta/classes/toolchain-scripts.bbclass     | 23 +++++++-----
 meta/lib/oeqa/sdk/cases/buildepoxy.py      |  3 +-
 meta/lib/oeqa/sdk/cases/buildgalculator.py |  3 +-
 meta/lib/oeqa/sdk/cases/python.py          | 11 ------
 meta/lib/oeqa/sdk/testmetaidesupport.py    | 43 ++++++++++++++++++++++
 meta/lib/oeqa/selftest/cases/meta_ide.py   | 14 +++++--
 meta/recipes-core/meta/meta-ide-support.bb | 26 +++++++++++--
 7 files changed, 94 insertions(+), 29 deletions(-)
 create mode 100644 meta/lib/oeqa/sdk/testmetaidesupport.py

Hi Alex

On Wednesday, 22 June 2022 22:33:06 NZDT Alexander Kanavin wrote:
> There's been a recent discussion about how we can make the Yocto SDK
> experience better [1]. One of the ideas was to eliminate the SDK
> as a separate artefact altogether and simply provide everything
> that the SDK and eSDKs do directly in a yocto build. This does not
> mean that people have to 'learn Yocto', but rather that the integrators
> should provide a well-functioning sstate cache infrastructure (same as
> with minimal eSDK, really), and a few wrapper scripts for setting up the
> build and the SDK environment that run layer setup and bitbake behind the
> scenes.

FWIW I think this is quite clever - well done :)

One question though:

> 2. Run:
> $ bitbake meta-ide-support
> $ bitbake -c populate_sysroot gtk+3
> (or any other target or native item that the application developer would
> need) 
> $ bitbake populate-sysroots

Should this have been "build-sysroots"? "populate-sysroots" doesn't seem to be 
a valid target. It looks like this is in the SDK manual as well.

Cheers
Paul

On Mon, 10 Oct 2022 at 10:23, Paul Eggleton
<bluelightning@bluelightning.org> wrote:
> > 2. Run:
> > $ bitbake meta-ide-support
> > $ bitbake -c populate_sysroot gtk+3
> > (or any other target or native item that the application developer would
> > need)
> > $ bitbake populate-sysroots
>
> Should this have been "build-sysroots"? "populate-sysroots" doesn't seem to be
> a valid target. It looks like this is in the SDK manual as well.

Yes, of course. It's a simple typo, I'll fix the docs.

Alex

Hello Alexander,

On Wed, Jun 22, 2022 at 12:33 PM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
>
> There's been a recent discussion about how we can make the Yocto SDK
> experience better [1]. One of the ideas was to eliminate the SDK
> as a separate artefact altogether and simply provide everything
> that the SDK and eSDKs do directly in a yocto build.
>
> So without further ado, here's how you get a 'SDK' with this set of patches:
>
Thanks for this work!  I like this approach.

Regards,

Leon.

Cheers - try the new layer setup and config management tools too :-)

Alex

On Thu, 13 Oct 2022 at 10:44, Leon Woestenberg <leon@sidebranch.com> wrote:
>
> Hello Alexander,
>
> On Wed, Jun 22, 2022 at 12:33 PM Alexander Kanavin
> <alex.kanavin@gmail.com> wrote:
> >
> > There's been a recent discussion about how we can make the Yocto SDK
> > experience better [1]. One of the ideas was to eliminate the SDK
> > as a separate artefact altogether and simply provide everything
> > that the SDK and eSDKs do directly in a yocto build.
> >
> > So without further ado, here's how you get a 'SDK' with this set of patches:
> >
> Thanks for this work!  I like this approach.
>
> Regards,
>
> Leon.