From patchwork Tue Sep 5 16:51:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41CC1CA1002 for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by mx.groups.io with SMTP id smtpd.web10.27268.1693933174362098010 for ; Tue, 05 Sep 2023 09:59:34 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=m6kGHog5; spf=pass (domain: gmail.com, ip: 209.85.208.169, mailfrom: adrian.freihofer@gmail.com) Received: by mail-lj1-f169.google.com with SMTP id 38308e7fff4ca-2bccda76fb1so45094871fa.2 for ; Tue, 05 Sep 2023 09:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933172; x=1694537972; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rUlTsoCRb7c61ci7ZVsjEUWCfwa7t3TQHbk7FZjqxn4=; b=m6kGHog5wvviT6RWxf2rOstCHV5ja1faUyMUZU/hcZLqPCLxW2m2zpbdo0N4f4IR3c VDR4mutdSMPcY2kaLE9Ilizqe09QwGLEZ2M2GB1BQ0Du3gUBRK7t1A+/GOEFTsglbUlT jlWfo39YOHUxBkT0y4jVJ9HiQj5hkr/f+piMTmhJgidCw6A6PgFscs0NV5TWH9T3nlBS vl8knIqWEByos07HpDXSDsiaiZyKnTh+eRoVs16HqVyfKALtuRlY6DrP0F4DmRPsFJJw wmaqo9+ycxLv3cwXLdqiM50pLzlEKV69HHLFP6lweRWDHmnwo6593Ae19GYAt5kbQdf8 c84Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933172; x=1694537972; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rUlTsoCRb7c61ci7ZVsjEUWCfwa7t3TQHbk7FZjqxn4=; b=ajvpsSKFe0Za0ecP/YZQrRxjWDpY3emttcARMw3pltHW69d9iBUbIx5S9BNvyU04sc eSVzExSfj1NKn47VrSYd6kyaZrJK/3Y6k4tCaJK6EUw2rRNLGC1XNM5GZxyv4nyb0gwT DT4y3kOYZL2nfM46xtNIAuq1vZh93VzuexZLB166ry8vbfhdYxti6zBbgoJ+8GC/7PCQ aLp8rDBewubO61HWkGpD/2ibFor+TjJTIgMKlM0m1cpLbolrmH1fghGtvYDqwigLG/8c 0DxYI/doZ3mJgfmAI++OfAO7dx0YqRINYN7zIEs0U7/0Rrc/hGrEHuZ3CKj9ZnigN/9q VJrg== X-Gm-Message-State: AOJu0YyW583I8MDpOtU82d0mavFKlHpjCYA6wLbtD9mIE8qPQIC6tvWA yKYFHbxMnFuVt+zRYKqAKjqvkjMii9U= X-Google-Smtp-Source: AGHT+IGcV4F9Q6z8yne9uBGYX1KLD8GDURvC92MpEf3aMOIJKL8kWUQP+92yup8jL1wwfKsvPmBlEw== X-Received: by 2002:a2e:980d:0:b0:2bc:fd7c:cd6d with SMTP id a13-20020a2e980d000000b002bcfd7ccd6dmr292557ljj.4.1693933171946; Tue, 05 Sep 2023 09:59:31 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:31 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 01/14] vscode: add minimal configuration Date: Tue, 5 Sep 2023 18:51:10 +0200 Message-ID: <20230905165913.2560907-2-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187233 It is essential to configure VSCode indexer plugins to ignore the build folder of bitbake. Otherwise, the indexer plugins run with 100% CPU load until an OOM exception occurs. In practice, this makes VSCode more or less unusable for working with Yocto until a file like the one added by this commit is deployed before VSCode starts. From the user's point of view, it is not obvious why the system runs at 100% CPU load and eventually crashes. It is even more misleading that VSCode starts the indexers immediately, but does not stop or reconfigure them when the ignore list is updated. In practice, this means that every time the ignore list is changed, VSCode immediately starts indexing the build folder until the OOM exception stops it. Depending on the system's OOM handler, the entire build machine may crash. Particularly annoying is the Python plugin that ignores the general ignore list and requires an extra ignore section. The settings are suitable for workflows like bitbake, devtool modify, devtool reset. The settings are not intended to work on the source code of a recipe. It is assumed that a separate instance of VSCode is used per workspace folder. These per workspace instances can have different settings depending on the details of the sources that come with the recipe. The new devtool ide plugin will generate settings to match this. Signed-off-by: Adrian Freihofer --- .gitignore | 2 ++ .vscode/settings.json | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 .vscode/settings.json diff --git a/.gitignore b/.gitignore index 8f48d452dab..f6ce090b5fc 100644 --- a/.gitignore +++ b/.gitignore @@ -36,3 +36,5 @@ _toaster_clones/ downloads/ sstate-cache/ toaster.sqlite +.vscode/ +vscode-bitbake-build/ diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 00000000000..517a86d1bfa --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,32 @@ +{ + "files.watcherExclude": { + "**/.git/**": true, + "**/cache/**": true, + "**/tmp*/**": true, + "**/downloads/**": true, + "**/sstate-cache/**": true, + "**/vscode-bitbake-build/**": true, + "**/workspace/sources/**": true, + "**/workspace/attic/**": true + }, + "files.exclude": { + "**/.git/**": true, + "**/cache/**": true, + "**/tmp*/**": true, + "**/downloads/**": true, + "**/sstate-cache/**": true, + "**/vscode-bitbake-build/**": true, + "**/workspace/sources/**": true, + "**/workspace/attic/**": true + }, + "python.analysis.exclude": [ + "**/.git/**", + "**/cache/**", + "**/tmp*/**", + "**/downloads/**", + "**/sstate-cache/**", + "**/vscode-bitbake-build/**", + "**/workspace/sources/**", + "**/workspace/attic/**" + ] +} From patchwork Tue Sep 5 16:51:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49600CA1004 for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mx.groups.io with SMTP id smtpd.web10.27269.1693933174933087128 for ; Tue, 05 Sep 2023 09:59:35 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=lkPNSy+y; spf=pass (domain: gmail.com, ip: 209.85.128.47, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-401b0d97850so28958165e9.2 for ; Tue, 05 Sep 2023 09:59:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933173; x=1694537973; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fsYHOSA9LHD1Fi0M0tsmfeVBU63JK+QmlGbN1QhUnDU=; b=lkPNSy+yJ3omHYZ6rLQpmDg6FDXcPcxcVYdTUh2AULuXZNxZz+hAYeXKYRGZJ6muJQ gCJfteWPcbTz70j9fym4HEgVjWA4CxNaP4HcxdMYAavZ3kiSMOBCw3ts/BKTyK55MPrk mSrMimd/x20ovpe3adH/ExIPh03B0bwAoKdPrRu8rB4yDvOdCb3fsuauwoRWcdV62gwK jW9pzQehnrFTadA+mWPwAUolbx4J9Irlj3oLXQoU1UHm0/oAl2R2Q3zbrWf5zyscIWra JN4vhyqId0EfGqck4GrUkESqoHoeUBopAPLd7+dKMu5F0Bx/oRIp6JoM3c5Z5XETOOTQ 3tAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933173; x=1694537973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fsYHOSA9LHD1Fi0M0tsmfeVBU63JK+QmlGbN1QhUnDU=; b=dkoiJSZGj0u5USi/B7YVqst9Iv8GFjic4eSEsgEzlz712qsbg+ZDKSmeYS5dT3dPdV oHVHE+TW8OLdI4fevLoVqTKRRey7tIYH/C7qMjpJTVOAicP0zwck8R71y7QyI2V9GdRf V96ju2dKr0zgwZsZic0RWseEuuuhiq3Yl4bHKaDtwX1BeNNiBXGTHBkGlTd5BYHWvl4w PZbogpBJAmt5qMhqiYrhClowDqNsqsLiIjkqSZn99ybQp+ll+PNXG9reeFUp2BMVoyrN c4+acX9tkwQV6oVjZMXm48VGZBiKo6SOVAm2DYPknjh0fxWjyzPbesvi5N/zvDKO0bsb 8G5w== X-Gm-Message-State: AOJu0Yx4bjL9ywnYsDmBphOQwf4v8FkH/PwJJ494i5id9+5a2H622Mss 5x7GgRqvcpIG/YTITjjwLaHa1WDmV6o= X-Google-Smtp-Source: AGHT+IHBphMzF97GfHP/ZtgbHOGB7ujLyC73tlklCtHypeEysexcyNZMaIOnTvSqT6D48lbPmwpapg== X-Received: by 2002:adf:e0c5:0:b0:31a:e73f:3fe7 with SMTP id m5-20020adfe0c5000000b0031ae73f3fe7mr305185wri.3.1693933172535; Tue, 05 Sep 2023 09:59:32 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:32 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 02/14] cmake.bbclass: refactor cmake args Date: Tue, 5 Sep 2023 18:51:11 +0200 Message-ID: <20230905165913.2560907-3-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187234 Make the details of the cmake configure arguments available to d.getVar(). This allows to share them with devtool via tinfoil. Signed-off-by: Adrian Freihofer --- meta/classes-recipe/cmake.bbclass | 43 +++++++++++++++++-------------- 1 file changed, 24 insertions(+), 19 deletions(-) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index ea836d5c0d9..69d566d3154 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -158,6 +158,29 @@ CONFIGURE_FILES = "CMakeLists.txt" do_configure[cleandirs] = "${@d.getVar('B') if d.getVar('S') != d.getVar('B') else ''}" +OECMAKE_ARGS = "\ + -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \ + -DCMAKE_INSTALL_BINDIR:PATH=${@os.path.relpath(d.getVar('bindir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_SBINDIR:PATH=${@os.path.relpath(d.getVar('sbindir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_LIBEXECDIR:PATH=${@os.path.relpath(d.getVar('libexecdir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_SYSCONFDIR:PATH=${sysconfdir} \ + -DCMAKE_INSTALL_SHAREDSTATEDIR:PATH=${@os.path.relpath(d.getVar('sharedstatedir'), d. getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_LOCALSTATEDIR:PATH=${localstatedir} \ + -DCMAKE_INSTALL_LIBDIR:PATH=${@os.path.relpath(d.getVar('libdir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_INCLUDEDIR:PATH=${@os.path.relpath(d.getVar('includedir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'), d.getVar('prefix') + '/')} \ + -DPYTHON_EXECUTABLE:PATH=${PYTHON} \ + -DPython_EXECUTABLE:PATH=${PYTHON} \ + -DPython3_EXECUTABLE:PATH=${PYTHON} \ + -DLIB_SUFFIX=${@d.getVar('baselib').replace('lib', '')} \ + -DCMAKE_INSTALL_SO_NO_EXE=0 \ + -DCMAKE_TOOLCHAIN_FILE:FILEPATH=${WORKDIR}/toolchain.cmake \ + -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \ + -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON \ + -DFETCHCONTENT_FULLY_DISCONNECTED=ON \ + -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON \ +" + cmake_do_configure() { if [ "${OECMAKE_BUILDPATH}" ]; then bbnote "cmake.bbclass no longer uses OECMAKE_BUILDPATH. The default behaviour is now out-of-tree builds with B=WORKDIR/build." @@ -178,25 +201,7 @@ cmake_do_configure() { ${OECMAKE_GENERATOR_ARGS} \ $oecmake_sitefile \ ${OECMAKE_SOURCEPATH} \ - -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \ - -DCMAKE_INSTALL_BINDIR:PATH=${@os.path.relpath(d.getVar('bindir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_SBINDIR:PATH=${@os.path.relpath(d.getVar('sbindir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_LIBEXECDIR:PATH=${@os.path.relpath(d.getVar('libexecdir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_SYSCONFDIR:PATH=${sysconfdir} \ - -DCMAKE_INSTALL_SHAREDSTATEDIR:PATH=${@os.path.relpath(d.getVar('sharedstatedir'), d. getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_LOCALSTATEDIR:PATH=${localstatedir} \ - -DCMAKE_INSTALL_LIBDIR:PATH=${@os.path.relpath(d.getVar('libdir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_INCLUDEDIR:PATH=${@os.path.relpath(d.getVar('includedir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'), d.getVar('prefix') + '/')} \ - -DPYTHON_EXECUTABLE:PATH=${PYTHON} \ - -DPython_EXECUTABLE:PATH=${PYTHON} \ - -DPython3_EXECUTABLE:PATH=${PYTHON} \ - -DLIB_SUFFIX=${@d.getVar('baselib').replace('lib', '')} \ - -DCMAKE_INSTALL_SO_NO_EXE=0 \ - -DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \ - -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \ - -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON \ - -DFETCHCONTENT_FULLY_DISCONNECTED=ON \ + ${OECMAKE_ARGS} \ ${EXTRA_OECMAKE} \ -Wno-dev } From patchwork Tue Sep 5 16:51:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41738CA1003 for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mx.groups.io with SMTP id smtpd.web11.27499.1693933175287912885 for ; Tue, 05 Sep 2023 09:59:35 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=HZXiZdcN; spf=pass (domain: gmail.com, ip: 209.85.221.53, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-317c3ac7339so2403539f8f.0 for ; Tue, 05 Sep 2023 09:59:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933173; x=1694537973; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CyVpYHI4gVYArcgKBwkHnDQB/WM8ccH2mqHOGr+UbCw=; b=HZXiZdcN+2eU/24c3em2cYjX3RX4L9tKMH+NnS8S4tHmGC7BUK+l+SAULwmoX47swS 7gbHyfE2VFE4B62cTU5YM/1ww9N9oZszOGYzYA6A64Nfr41pV9A0/dNg1SZciNEp8xyj NR+rBvuDoTgRd1NAQ8zdFnuHZ3C2SbFpoAa8dAozreXn2MfVGXzd+E7L7/jLXnczznjB orASctlLdxWRn6IcHlEIq2Y5Bkx8dWy+xRGGPSlJ+5tLOToNyZMmGGb++EQlwsiibTRV CM5x7iffvEG4YIcOPY74cwylKI+E805bvKtOgub4UvESVaRjhgdj6Bqanzyiy6yZoNON hVzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933173; x=1694537973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CyVpYHI4gVYArcgKBwkHnDQB/WM8ccH2mqHOGr+UbCw=; b=KOvE2llXl4MePrJLkJUDsqt6EVtB8b6ABehqByDSaX4mjpv+qAOde0/N1OInRgCct0 QHULBijXKU5Y4wbl6Os8JdbNC09NJz8HClQbebLYmWEeAzov16RnQUdoWyb+vqbzImbi 05hqBokkop6B07rzbKiV3e6EOWrqzAK7VDjmZXTkSPJ0a1fg9yvz3/TiYqajf/UM2Cwh tOFXifm9YyctSrXHsYZv3KK5n1dOZE2b5fHPkTaxtm7uRikhE2MuD1nR466Y30FoSobN 6jAvcljIuRoZweeAXrjZT5i+J55Q0oQSIAZxuw2VIn2kuP0RG4jWXxFsacGBG7RMO0WT iHRg== X-Gm-Message-State: AOJu0YyjmSSUXZWY9M9QZukhUCknyycC2PyRNLK3yCx0C/kRa0oOh6cG +lol94lBTwVJLQe4Z2lkrWIdj8KcsrA= X-Google-Smtp-Source: AGHT+IGqRAZ8QT95cn6DrpcHcQyJ0LsdbN6ICDK2KYreSOsjG8CPe5UNNiJeQZ2ZMPmaQRxzrOmcLA== X-Received: by 2002:a5d:4538:0:b0:31c:e933:9590 with SMTP id j24-20020a5d4538000000b0031ce9339590mr303296wra.33.1693933173090; Tue, 05 Sep 2023 09:59:33 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:32 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 03/14] cmake.bbclass: cleanup spaces and tabs Date: Tue, 5 Sep 2023 18:51:12 +0200 Message-ID: <20230905165913.2560907-4-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187235 Signed-off-by: Adrian Freihofer --- meta/classes-recipe/cmake.bbclass | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index 69d566d3154..41748b08207 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -90,12 +90,14 @@ def map_host_arch_to_uname_arch(host_arch): return "ppc64" return host_arch + cmake_do_generate_toolchain_file() { if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then cmake_crosscompiling="set( CMAKE_CROSSCOMPILING FALSE )" - else - cmake_sysroot="set( CMAKE_SYSROOT \"${RECIPE_SYSROOT}\" )" + else + cmake_sysroot="set( CMAKE_SYSROOT \"${RECIPE_SYSROOT}\" )" fi + cat > ${WORKDIR}/toolchain.cmake < X-Patchwork-Id: 29999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31D9BCA1001 for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by mx.groups.io with SMTP id smtpd.web10.27270.1693933175706388320 for ; Tue, 05 Sep 2023 09:59:36 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=USDUtvQt; spf=pass (domain: gmail.com, ip: 209.85.128.49, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-402c46c49f4so27196405e9.1 for ; Tue, 05 Sep 2023 09:59:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933174; x=1694537974; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MND5e5KB4OZVVC644V91vwSuJuyFeLqIvR4uWSLBv8M=; b=USDUtvQtD16ql6FqCdlDZ4IwVvvjXxeuGvFK8RLeycYPh/8/UATlrxvHQViPGyVit9 IT4j/ScyCM9m3oOlq9Pp2wrR202HlBvORq5yO1m5vOSFA7QO2wfsbTZ7ZWablqg2Uvjg KLz+VwvLOVRvlWPZAMoRwPGbAXfkAMgNHK3HqYMefIOdZ60USpODLzRouA70+Bb1rkNV iOVMlTr29Zivj+Qaz2V/IyxD6o+iSjLmv4SMLONOVXWKr5BDjhPMOSQCAolg+oLHEEt2 h+4MPXFzIPRD+ENRGC/LMiqUbFodMK6QSNaSMS7sLpwaKDOEn0cekgV1Qhm1iCssALcj pRJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933174; x=1694537974; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MND5e5KB4OZVVC644V91vwSuJuyFeLqIvR4uWSLBv8M=; b=JsSbr37rhDvwVirrWdJi5VEldNNHrcYFBXz2lyIKbB1f1ZO9yYu959ViOj4XnnG8ad laLyPE8DK7t2UUrXXLuPfiwZnzxaMIzfqY+aDSo5Tfcs1otOhJBheJOayrGGGvK4+RbI c77EZPiXasXKe48Txf79RuVwl35ebB4fZmtF5avmkUH9eGYmuCjGWHub+OmxmCS07kri XJcxkQLi5DVFwrpHEFDkuw5RzS1YdDpPF7JRh2QJqX02qrvJHkUDpmaJIWlaq6Yus/55 jfPPyYRvAV+9xMYmVjwWRycvQ/ETmMkRt1odMVjgKnxqDBtA1UXDMRYHo/3z90SjkxjW FfkA== X-Gm-Message-State: AOJu0YzDKSl6tLERi3+Ex/Y3mP3YfvyApFVWxoHtzs+PZzyIQbfIOeAO wu2Zq3C4RgJAgKG76yiTIZYHyfvEL0Q= X-Google-Smtp-Source: AGHT+IELhxbbFhMS2Ulgp3bvPQir6oUaGUYP7t4Vj49T8MkpMAMio0dYisiFj4COveTUSM0AkVLamA== X-Received: by 2002:adf:f3ca:0:b0:317:57f0:fae with SMTP id g10-20020adff3ca000000b0031757f00faemr299472wrp.63.1693933173837; Tue, 05 Sep 2023 09:59:33 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:33 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 04/14] cmake.bbclass: support qemu Date: Tue, 5 Sep 2023 18:51:13 +0200 Message-ID: <20230905165913.2560907-5-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187236 Define the CMAKE_CROSSCOMPILING_EMULATOR variable similar to what the meson bbclass does. This allows for example to execute cross compilied unit tests on the build machine. CMAKE_CROSSCOMPILING_EMULATOR is a semi colon separated list of paramters which could directly handle the -L and the -E parameters. Creating a wrapper script is not absolutely mandatory. But anyway lets do it similar to what the meson.bbclass does and also disable pseudo. Signed-off-by: Adrian Freihofer --- meta/classes-recipe/cmake.bbclass | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index 41748b08207..68b4e83bbe8 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -4,6 +4,13 @@ # SPDX-License-Identifier: MIT # +inherit qemu + +EXEWRAPPER_ENABLED:class-native = "False" +EXEWRAPPER_ENABLED:class-nativesdk = "False" +EXEWRAPPER_ENABLED ?= "${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'True', 'False', d)}" +DEPENDS:append = "${@' qemu-native' if d.getVar('EXEWRAPPER_ENABLED') == 'True' else ''}" + # Path to the CMake file to process. OECMAKE_SOURCEPATH ??= "${S}" @@ -156,6 +163,20 @@ EOF addtask generate_toolchain_file after do_patch before do_configure +cmake_do_generate_toolchain_file:append:class-target() { + if [ "${EXEWRAPPER_ENABLED}" = "True" ]; then + # Write out a qemu wrapper that will be used as exe_wrapper so that camake + # can run target helper binaries through that. This also allows to execute ctest. + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_HOST}', ['${STAGING_DIR_HOST}/${libdir}','${STAGING_DIR_HOST}/${base_libdir}'])}" + echo "#!/bin/sh" > "${WORKDIR}/cmake-qemuwrapper" + echo "$qemu_binary \"\$@\"" >> "${WORKDIR}/cmake-qemuwrapper" + chmod +x "${WORKDIR}/cmake-qemuwrapper" + echo "set( CMAKE_CROSSCOMPILING_EMULATOR ${WORKDIR}/cmake-qemuwrapper)" \ + >> ${WORKDIR}/toolchain.cmake + fi +} + + CONFIGURE_FILES = "CMakeLists.txt" do_configure[cleandirs] = "${@d.getVar('B') if d.getVar('S') != d.getVar('B') else ''}" From patchwork Tue Sep 5 16:51:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31602CA0FFD for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by mx.groups.io with SMTP id smtpd.web10.27271.1693933176940574840 for ; Tue, 05 Sep 2023 09:59:37 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=k3IB/qjz; spf=pass (domain: gmail.com, ip: 209.85.221.44, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f44.google.com with SMTP id ffacd0b85a97d-31dd10c2b8bso2291418f8f.3 for ; Tue, 05 Sep 2023 09:59:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933175; x=1694537975; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WbP3FzRf7OkH8/a1cCRReLeQdZU4BRcOP/pO5az/dPg=; b=k3IB/qjzfebPsCdyWbeH12mU9MhNQZB++9h6my9W/rIcCwvQseulsOZcKwKI7CP5zN Yjgz38RE5qq90Aifd7s8nRdwxXV5ijM5L7npNDrw4NDCWkvGlWrESyCpNB5o5Nu61szU zHIHY8uWXnfz6Xz5EuX1SU+D0zWh3klhKKOiz2IJeLQvTOtm4BDLVjXvNgGinOxnSppK 8n620hsim/E8onYoD8DwDGczRI+ZaGY/ZkKqyUmG8TpH0ZOXTdoCgLmwiWUoFkfDefJ0 UIcprkxkhMUVmNl4/BWRR6kUn60S0lfHQFnLDSTnDn7BNGRDNqGpVzySVuRwSnv3uloj bX1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933175; x=1694537975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WbP3FzRf7OkH8/a1cCRReLeQdZU4BRcOP/pO5az/dPg=; b=MBhU9gvX7TEz6oDAZcHUDTUWTsGmXoK3WYDagfMGo/6/y4GT9VIZMzHkvPLudMGCaE HaINwG4LyFz6GRigxOqgXlkxLhxURf3a8dhigU4EPPNOO/dKxZJrXrCo+mqLlr7HdBpr NftrQ/W7jYibzYjRiGAWer5QeLbWmvPhYfjeu4TuNUbNyURXx5bGGnzfFbUT6Ry6F8Sz xC2Ug+wNE7UM39Sv9VPnqjpXdTym+fELR1M+7h6ZAbtGPFUkmzkpOgtiCzthUnEYgOOy Or34txFZqDHiM6GutFAnDBQLT6919V7kYnccQ7bXPpJNDI8CUZc3EBW7CuCM8J8hMv2q El1A== X-Gm-Message-State: AOJu0YxGv+QzPOLCwjtvRVntccmgKiFqLq2WI9pyjRKXozXn/idx4spJ KR72Vg2gFbBfLrlXQ+SjdhW7cpF7V20= X-Google-Smtp-Source: AGHT+IGR65EmOuPNBr5kFE856Z6JziFfT1bDbFU+/rFoWU1B00fdOZ5ZH8pgALVQObln8yPUamieoA== X-Received: by 2002:a05:6000:10c3:b0:317:6965:ef4d with SMTP id b3-20020a05600010c300b003176965ef4dmr305604wrx.56.1693933174722; Tue, 05 Sep 2023 09:59:34 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:34 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 05/14] devtool: new ide plugin Date: Tue, 5 Sep 2023 18:51:14 +0200 Message-ID: <20230905165913.2560907-6-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187237 The new devtool ide plugin configures an IDE to work with the eSDK. With this initial implementation VSCode is the default IDE. The plugin works for recipes inheriting the cmake or the meson bbclass. Support for more programming languages and build tools may be added in the future. Using the plugin in recipe modes: $ devtool modify a-recipe $ devtool ide a-recipe a-image $ code "$BUILDDIR/workspace/sources/a-recipe" Work in VSCode, after installing the proposed plugins Using the plugin without a recipe $ devtool ide none a-image vscode where/the/sources/are Use the cross tool-chain which is provided as a cmake-kit. The goal of this implementation is to create a configuration for VSCode (or other IDEs) that allows to work on the code of a recipe completely independent from bitbake. bitbake is only called if the configuration or the whole SDK has to be regenerated. But bitbake should not need to be called while working in the IDE. This has two major advantages over calling devtool build from the IDE: - The IDE provides plugins for integration with cmake, for example. These features are usable, which would not be the case if bitbake or devtool are called from within the IDE. - It is much faster. Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide.py | 1190 ++++++++++++++++++++++++++++++++++++ 1 file changed, 1190 insertions(+) create mode 100755 scripts/lib/devtool/ide.py diff --git a/scripts/lib/devtool/ide.py b/scripts/lib/devtool/ide.py new file mode 100755 index 00000000000..7c7040232c4 --- /dev/null +++ b/scripts/lib/devtool/ide.py @@ -0,0 +1,1190 @@ +#! /usr/bin/env python3 +# +# Copyright (C) 2023 Siemens AG +# +# SPDX-License-Identifier: GPL-2.0-only +# + +"""Devtool ide plugin""" + +import os +import stat +import logging +import json +import re +import shutil +from argparse import RawTextHelpFormatter +from enum import IntEnum, auto + +import bb +from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, DevtoolError, parse_recipe +from devtool.standard import get_real_srctree + +SHARED_SYSROOT_RECIPES = ['none', 'meta-ide-support', 'build-sysroots'] +SUPPORTED_IDES = ['code', 'none'] + +logger = logging.getLogger('devtool') + + +class TargetDevice: + """SSH remote login parameters""" + + def __init__(self, args): + self.extraoptions = '' + if args.no_host_check: + self.extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' + self.ssh_sshexec = 'ssh' + if args.ssh_exec: + self.ssh_sshexec = args.ssh_exec + self.ssh_port = '' + if args.port: + self.ssh_port = "-p %s" % args.port + if args.key: + self.extraoptions += ' -i %s' % args.key + + self.target = args.target + target_sp = args.target.split('@') + if len(target_sp) == 1: + self.login = "" + self.host = target_sp[0] + elif len(target_sp) == 2: + self.login = target_sp[0] + self.host = target_sp[1] + else: + logger.error("Invalid target argument: %s" % args.target) + + +class RecipeNative: + def __init__(self, name, target_arch=None): + self.name = name + self.target_arch = target_arch + self.bootstrap_tasks = [self.name + ':do_addto_recipe_sysroot'] + self.staging_bindir_native = None + self.target_sys = None + self.__native_bin = None + + def initialize(self, config, workspace, tinfoil): + recipe_d = parse_recipe( + config, tinfoil, self.name, appends=True, filter_workspace=False) + if not recipe_d: + raise DevtoolError("Parsing %s recipe failed" % self.name) + self.staging_bindir_native = os.path.realpath( + recipe_d.getVar('STAGING_BINDIR_NATIVE')) + self.target_sys = recipe_d.getVar('TARGET_SYS') + + @property + def native_bin(self): + if not self.__native_bin: + raise DevtoolError("native binary name is not defined.") + return self.__native_bin + + +class RecipeGdbCross(RecipeNative): + def __init__(self, args, target_arch, target_device, gdbserver_multi=True): + super().__init__('gdb-cross-' + target_arch, target_arch) + self.target_device = target_device + self.gdb = None + self.gdbserver_port_next = int(args.gdbserver_port_start) + self.gdbserver_multi = gdbserver_multi + self.config_db = {} + + def initialize(self, config, workspace, tinfoil): + super().initialize(config, workspace, tinfoil) + gdb_bin = self.target_sys + '-gdb' + gdb_path = os.path.join( + self.staging_bindir_native, self.target_sys, gdb_bin) + self.gdb = gdb_path + + @property + def host(self): + return self.target_device.host + + def __gdbserver_start_cmd(self, binary, port): + if self.gdbserver_multi: + gdbserver_cmd = "/usr/bin/gdbserver --multi :%s" % ( + port) + else: + gdbserver_cmd = "/usr/bin/gdbserver --once :%s %s" % ( + port, binary) + return "%s %s %s %s 'sh -c \"%s\"'" % ( + self.target_device.ssh_sshexec, self.target_device.ssh_port, self.target_device.extraoptions, self.target_device.target, gdbserver_cmd) + + def setup_gdbserver_config(self, binary, script_dir): + if binary in self.config_db: + raise DevtoolError( + "gdbserver config for binary %s is already generated" % binary) + + port = self.gdbserver_port_next + self.gdbserver_port_next += 1 + config_entry = { + "port": port, + } + if script_dir: + cmd_lines = ['#!/bin/sh'] + cmd_lines.append(self.__gdbserver_start_cmd(binary, port)) + binary_name_pretty = binary.replace(os.sep, '-') + start_script_name = 'gdbserver_start_%d_%s' % ( + port, binary_name_pretty) + if self.gdbserver_multi: + start_script_name += "_m" + start_script_file = os.path.join(script_dir, start_script_name) + config_entry['gdbserver_start_script'] = start_script_file + config_entry['pretty_id'] = 'gdbserver start %d %s' % ( + port, binary) + + bb.utils.mkdirhier(script_dir) + with open(start_script_file, 'w') as script_f: + script_f.write(os.linesep.join(cmd_lines)) + script_f.write(os.linesep) + st = os.stat(start_script_file) + os.chmod(start_script_file, st.st_mode | stat.S_IEXEC) + + self.config_db[binary] = config_entry + return config_entry + + def get_gdbserver_start_scripts(self): + for conf in self.config_db.values(): + yield (conf['pretty_id'], conf['gdbserver_start_script']) + + def get_gdbserver_pretty_id(self, binary): + return self.config_db[binary]['pretty_id'] + + def get_gdbserver_port(self, binary): + return self.config_db[binary]['port'] + + def get_gdbserver_start_script(self, binary): + return self.config_db[binary]['gdbserver_start_script'] + + +class RecipeImage: + """Handle some image recipe related properties""" + + def __init__(self, name): + self.combine_dbg_image = False + self.debuginfod = False + self.name = name + self.package_debug_split_style = None + self.rootfs = None + self.__rootfs_dbg = None + self.bootstrap_tasks = [self.name + ':do_build'] + + def initialize(self, config, tinfoil): + image_d = parse_recipe( + config, tinfoil, self.name, appends=True, filter_workspace=False) + if not image_d: + raise DevtoolError( + "Parsing image recipe %s failed" % self.name) + if 'debuginfod' in image_d.getVar('DISTRO_FEATURES').split(): + # image_config.debuginfod = True + logger.warning("Support for debuginfod is not implemented yet.") + + self.package_debug_split_style = image_d.getVar( + 'PACKAGE_DEBUG_SPLIT_STYLE') + + workdir = image_d.getVar('WORKDIR') + self.rootfs = os.path.join(workdir, 'rootfs') + if image_d.getVar('IMAGE_GEN_DEBUGFS') == "1": + self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg') + + self.combine_dbg_image = bb.data.inherits_class( + 'image-combined-dbg', image_d) + + @property + def debug_support(self): + return bool(self.debuginfod or self.rootfs_dbg) + + @property + def rootfs_dbg(self): + if self.__rootfs_dbg and os.path.isdir(self.__rootfs_dbg): + return self.__rootfs_dbg + return None + + def solib_search_path_rootfs(self): + """Search for folders with shared libraries in the rootfs and rootfs-dbg + + This is based on the assumption that the PACKAGE_DEBUG_SPLIT_STYLE variable from the image + is the global setting which is used by most packages. Even if this variable does not seem + to make sense in the image context. + """ + rootfs_solib_search_path = [] + rootfs_dbg_solib_search_path = [] + if self.package_debug_split_style in ['debug-with-srcpkg', '.debug']: + if self.combine_dbg_image: + rootfs_dbg_solib_search_path = [ + "/lib", "/lib/.debug", "/usr/lib", "/usr/lib/.debug"] + else: + logger.warn( + 'Adding IMAGE_CLASSES += "image-combined-dbg" offers better remote debugging experience.') + rootfs_solib_search_path = [ + "/lib", "/usr/lib"] + rootfs_dbg_solib_search_path = [ + "/lib/.debug", "/usr/lib/.debug"] + elif self.package_debug_split_style == 'debug-file-directory': + rootfs_dbg_solib_search_path = ["/usr/lib/debug"] + else: + logger.warning( + "Cannot find solib search path for a rootfs built with PACKAGE_DEBUG_SPLIT_STYLE=%s." % self.package_debug_split_style) + + sym_dirs = [] + for dbgdir in rootfs_solib_search_path: + sym_dirs.append(os.path.join( + self.rootfs, dbgdir.lstrip('/'))) + for dbgdir in rootfs_dbg_solib_search_path: + sym_dirs.append(os.path.join( + self.rootfs_dbg, dbgdir.lstrip('/'))) + + return sym_dirs + + +class RecipeMetaIdeSupport: + """Handle some meta-ide-support recipe related properties""" + + def __init__(self): + self.bootstrap_tasks = ['meta-ide-support:do_build'] + self.topdir = None + self.datadir = None + self.deploy_dir_image = None + self.build_sys = None + # From toolchain-scripts + self.real_multimach_target_sys = None + + def initialize(self, config, tinfoil): + meta_ide_support_d = parse_recipe( + config, tinfoil, 'meta-ide-support', appends=True, filter_workspace=False) + if not meta_ide_support_d: + raise DevtoolError("Parsing meta-ide-support recipe failed") + + self.topdir = meta_ide_support_d.getVar('TOPDIR') + self.datadir = meta_ide_support_d.getVar('datadir') + self.deploy_dir_image = meta_ide_support_d.getVar( + 'DEPLOY_DIR_IMAGE') + self.build_sys = meta_ide_support_d.getVar('BUILD_SYS') + self.real_multimach_target_sys = meta_ide_support_d.getVar( + 'REAL_MULTIMACH_TARGET_SYS') + + +class RecipeBuildSysroots: + def __init__(self): + self.standalone_sysroot = None + self.standalone_sysroot_native = None + self.bootstrap_tasks = ['build-sysroots:do_build'] + + def initialize(self, config, tinfoil): + build_sysroots_d = parse_recipe( + config, tinfoil, 'build-sysroots', appends=True, filter_workspace=False) + if not build_sysroots_d: + raise DevtoolError("Parsing build-sysroots recipe failed") + self.standalone_sysroot = build_sysroots_d.getVar( + 'STANDALONE_SYSROOT') + self.standalone_sysroot_native = build_sysroots_d.getVar( + 'STANDALONE_SYSROOT_NATIVE') + + +class SharedSysrootsEnv: + def __init__(self): + self.ide_support = None + self.build_sysroots = None + + def initialize(self, ide_support, build_sysroots): + self.ide_support = ide_support + self.build_sysroots = build_sysroots + + def __vscode_update_kits(self): + """Expose the toolchain of the dSDK""" + datadir = self.ide_support.datadir + deploy_dir_image = self.ide_support.deploy_dir_image + real_multimach_target_sys = self.ide_support.real_multimach_target_sys + standalone_sysroot_native = self.build_sysroots.standalone_sysroot_native + vscode_ws_path = os.path.join( + os.environ['HOME'], '.local', 'share', 'CMakeTools') + cmake_kits_path = os.path.join(vscode_ws_path, 'cmake-tools-kits.json') + oecmake_generator = "Ninja" + env_script = os.path.join( + deploy_dir_image, 'environment-setup-' + real_multimach_target_sys) + + logger.info("updating %s" % cmake_kits_path) + + if not os.path.isdir(vscode_ws_path): + os.makedirs(vscode_ws_path) + cmake_kits_old = [] + if os.path.exists(cmake_kits_path): + with open(cmake_kits_path, 'r', encoding='utf-8') as cmake_kits_file: + cmake_kits_old = json.load(cmake_kits_file) + cmake_kits = cmake_kits_old.copy() + + cmake_kit_new = { + "name": "OE " + real_multimach_target_sys, + "environmentSetupScript": env_script, + "toolchainFile": standalone_sysroot_native + datadir + "/cmake/OEToolchainConfig.cmake", + "preferredGenerator": { + "name": oecmake_generator + } + } + + def merge_kit(cmake_kits, cmake_kit_new): + i = 0 + while i < len(cmake_kits): + if 'environmentSetupScript' in cmake_kits[i] and \ + cmake_kits[i]['environmentSetupScript'] == cmake_kit_new['environmentSetupScript']: + cmake_kits[i] = cmake_kit_new + return + i += 1 + cmake_kits.append(cmake_kit_new) + merge_kit(cmake_kits, cmake_kit_new) + + if cmake_kits != cmake_kits_old: + bb.note("Updating: %s" % cmake_kits_path) + with open(cmake_kits_path, 'w', encoding='utf-8') as cmake_kits_file: + json.dump(cmake_kits, cmake_kits_file, indent=4) + + def setup_ide(self, args): + if args.ide == 'code': + self.__vscode_update_kits() + + +class BuildTool(IntEnum): + UNDEFINED = auto() + CMAKE = auto() + MESON = auto() + + +class RecipeModified: + + def __init__(self, name): + self.name = name + self.bootstrap_tasks = [name + ':do_install'] + # workspace + self.real_srctree = None + self.srctree = None + self.temp_dir = None + self.bbappend = None + # recipe variables from d.getVar + self.b = None + self.bpn = None + self.d = None + self.fakerootcmd = None + self.fakerootenv = None + self.package_arch = None + self.path = None + self.recipe_sysroot = None + self.recipe_sysroot_native = None + self.staging_incdir = None + self.target_arch = None + self.workdir = None + # replicate bitbake build environment + self.__exported_vars = None + self.cmd_compile = None + # main build tool used by this recipe + self.build_tool = BuildTool.UNDEFINED + # build_tool = cmake + self.oecmake_generator = None + self.__cmake_cache_vars = None + # build_tool = meson + self.meson_buildtype = None + self.meson_wrapper = None + self.mesonopts = None + self.extra_oemeson = None + self.meson_cross_file = None + # vscode + self.dot_code_dir = None + # TODO: remove calling devtool + self.bb_env_passthrough_additions = None + self.bbpath = None + self.bitbakepath = None + self.topdir = None + + def initialize(self, config, workspace, tinfoil): + recipe_d = parse_recipe( + config, tinfoil, self.name, appends=True, filter_workspace=False) + if not recipe_d: + raise DevtoolError("Parsing %s recipe failed" % self.name) + + # Verify this recipe is built as externalsrc setup by devtool modify + workspacepn = check_workspace_recipe( + workspace, self.name, bbclassextend=True) + self.srctree = workspace[workspacepn]['srctree'] + # Need to grab this here in case the source is within a subdirectory + self.real_srctree = get_real_srctree( + self.srctree, recipe_d.getVar('S'), recipe_d.getVar('WORKDIR')) + self.bbappend = workspace[workspacepn]['bbappend'] + + self.temp_dir = os.path.join(config.workspace_path, 'temp', self.name) + if os.path.exists(self.temp_dir): + shutil.rmtree(self.temp_dir) + + self.b = recipe_d.getVar('B') + self.bpn = recipe_d.getVar('BPN') + self.d = recipe_d.getVar('D') + self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD') + self.fakerootenv = recipe_d.getVar('FAKEROOTENV') + self.package_arch = recipe_d.getVar('PACKAGE_ARCH') + self.path = recipe_d.getVar('PATH') + self.recipe_sysroot = os.path.realpath( + recipe_d.getVar('RECIPE_SYSROOT')) + self.recipe_sysroot_native = os.path.realpath( + recipe_d.getVar('RECIPE_SYSROOT_NATIVE')) + self.staging_incdir = os.path.realpath( + recipe_d.getVar('STAGING_INCDIR')) + self.target_arch = recipe_d.getVar('TARGET_ARCH') + self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR')) + + self.bb_env_passthrough_additions = recipe_d.getVar( + 'BB_ENV_PASSTHROUGH_ADDITIONS') + self.bbpath = recipe_d.getVar('BBPATH') + self.bitbakepath = recipe_d.getVar('BITBAKEPATH') + self.topdir = recipe_d.getVar('TOPDIR') + + self.__init_exported_variables(recipe_d) + + if bb.data.inherits_class('cmake', recipe_d): + self.oecmake_generator = recipe_d.getVar('OECMAKE_GENERATOR') + self.__init_cmake_preset_cache(recipe_d) + self.build_tool = BuildTool.CMAKE + elif bb.data.inherits_class('meson', recipe_d): + self.meson_buildtype = recipe_d.getVar('MESON_BUILDTYPE') + self.mesonopts = recipe_d.getVar('MESONOPTS') + self.extra_oemeson = recipe_d.getVar('EXTRA_OEMESON') + self.meson_cross_file = recipe_d.getVar('MESON_CROSS_FILE') + self.build_tool = BuildTool.MESON + + self.dot_code_dir = os.path.join(self.srctree, '.vscode') + + def append_to_bbappend(self, append_text): + with open(self.bbappend, 'a') as bbap: + bbap.write(append_text) + + def remove_from_bbappend(self, append_text): + with open(self.bbappend, 'r') as bbap: + text = bbap.read() + new_text = text.replace(append_text, '') + with open(self.bbappend, 'w') as bbap: + bbap.write(new_text) + + def debug_build_config(self, args): + """Explicitely set for example CMAKE_BUILD_TYPE to Debug if not defined otherwise""" + if self.build_tool == BuildTool.CMAKE: + append_text = os.linesep + \ + 'OECMAKE_ARGS:append = " -DCMAKE_BUILD_TYPE:STRING=Debug"' + os.linesep + if args.debug_build_config and not 'CMAKE_BUILD_TYPE' in self.__cmake_cache_vars: + self.__cmake_cache_vars['CMAKE_BUILD_TYPE'] = { + "type": "STRING", + "value": "Debug", + } + self.append_to_bbappend(append_text) + elif 'CMAKE_BUILD_TYPE' in self.__cmake_cache_vars: + del self.__cmake_cache_vars['CMAKE_BUILD_TYPE'] + self.remove_from_bbappend(append_text) + elif self.build_tool == BuildTool.MESON: + append_text = os.linesep + 'MESON_BUILDTYPE = "debug"' + os.linesep + if args.debug_build_config and self.meson_buildtype != "debug": + self.mesonopts.replace( + '--buildtype ' + self.meson_buildtype, '--buildtype debug') + self.append_to_bbappend(append_text) + elif self.meson_buildtype == "debug": + self.mesonopts.replace( + '--buildtype debug', '--buildtype plain') + self.remove_from_bbappend(append_text) + elif args.debug_build_config: + logger.warn( + "--debug-build-config is not implemented for this build tool yet.") + + def solib_search_path_sysroot(self): + return [os.path.join(self.recipe_sysroot, p) for p in ['lib', 'usr/lib']] + + def solib_search_path(self, image): + return image.solib_search_path_rootfs() + self.solib_search_path_sysroot() + + def solib_search_path_str(self, image): + return ':'.join(self.solib_search_path(image)) + + def __init_exported_variables(self, d): + """Find all variables with export flag set.""" + exported_vars = {} + + vars = (key for key in d.keys() if not key.startswith( + "__") and not d.getVarFlag(key, "func", False)) + for var in vars: + func = d.getVarFlag(var, "func", False) + if d.getVarFlag(var, 'python', False) and func: + continue + export = d.getVarFlag(var, "export", False) + unexport = d.getVarFlag(var, "unexport", False) + if not export and not unexport and not func: + continue + if unexport: + continue + + val = d.getVar(var) + if val is None: + continue + if set(var) & set("-.{}+"): + logger.warn( + "Warning: Found invalid character in variable name %s", str(var)) + continue + varExpanded = d.expand(var) + val = str(val) + + if varExpanded.startswith("BASH_FUNC_"): + varExpanded = varExpanded[10:-2] + val = val[3:] # Strip off "() " + logger.warn("Warning: BASH_FUNC_ is not exported to cmake presets (%s() %s)" % ( + varExpanded, val)) + continue + + if func: + code_line = "line: {0}, file: {1}\n".format( + d.getVarFlag(var, "lineno", False), + d.getVarFlag(var, "filename", False)) + val = val.rstrip('\n') + logger.warn("Warning: exported shell function %s() is not exported (%s)" % + (varExpanded, code_line)) + continue + + if export: + exported_vars[varExpanded] = val.strip() + continue + + self.__exported_vars = exported_vars + + def __init_cmake_preset_cache(self, d): + """Replicate the cmake configure arguments with all details to share on build folder between bitbake and SDK.""" + site_file = os.path.join(self.workdir, 'site-file.cmake') + if os.path.exists(site_file): + print("Warning: site-file.cmake is not supported") + + cache_vars = {} + oecmake_args = d.getVar('OECMAKE_ARGS').split() + extra_oecmake = d.getVar('EXTRA_OECMAKE').split() + for param in oecmake_args + extra_oecmake: + d_pref = "-D" + if param.startswith(d_pref): + param = param[len(d_pref):] + else: + print("Error: expected a -D") + param_s = param.split('=', 1) + param_nt = param_s[0].split(':', 1) + + def handle_undefined_variable(var): + if var.startswith('${') and var.endswith('}'): + return '' + else: + return var + # Example: FOO=ON + if len(param_nt) == 1: + cache_vars[param_s[0]] = handle_undefined_variable(param_s[1]) + # Example: FOO:PATH=/tmp + elif len(param_nt) == 2: + cache_vars[param_nt[0]] = { + "type": param_nt[1], + "value": handle_undefined_variable(param_s[1]), + } + else: + print("Error: cannot parse %s" % param) + self.__cmake_cache_vars = cache_vars + + def __cmake_preset(self): + toolchain_file = os.path.join(self.workdir, 'toolchain.cmake') + preset_name = self.bpn + "-" + self.package_arch + preset_display_name = self.bpn + ": " + self.package_arch + cmake_executable = os.path.join( + self.recipe_sysroot_native, 'usr', 'bin', 'cmake') + self.cmd_compile = cmake_executable + " --build --preset " + preset_name + + preset_dict_configure = { + "name": preset_name, + "displayName": preset_display_name, + "description": "Bitbake build environment for the recipe %s compiled for %s" % (self.bpn, self.package_arch), + "binaryDir": self.b, + "generator": self.oecmake_generator, + "toolchainFile": toolchain_file, + "cacheVariables": self.__cmake_cache_vars, + "environment": self.__exported_vars, + "cmakeExecutable": cmake_executable + } + + preset_dict_build = { + "name": preset_name, + "displayName": preset_display_name, + "description": "Bitbake build environment for the recipe %s compiled for %s" % (self.bpn, self.package_arch), + "configurePreset": preset_name, + "inheritConfigureEnvironment": True + } + + preset_dict_test = { + "name": preset_name, + "displayName": preset_display_name, + "description": "Bitbake build environment for the recipe %s compiled for %s" % (self.bpn, self.package_arch), + "configurePreset": preset_name, + "inheritConfigureEnvironment": True + } + + preset_dict = { + "version": 3, # cmake 3.21, backward compatible with kirkstone + "configurePresets": [preset_dict_configure], + "buildPresets": [preset_dict_build], + "testPresets": [preset_dict_test] + } + + logger.info("generating cmake preset for recipe %s" % self.bpn) + preset_file = os.path.join(self.real_srctree, 'CMakeUserPresets.json') + with open(preset_file, 'w') as outfile: + json.dump(preset_dict, outfile, indent=4) + + @staticmethod + def update_json_file(dot_code_dir, json_file, update_dict): + json_path = os.path.join(dot_code_dir, json_file) + logger.info("Updating vscode %s (%s)" % (json_file, json_path)) + if not os.path.exists(dot_code_dir): + os.makedirs(dot_code_dir) + try: + with open(json_path) as f: + orig_dict = json.load(f) + except json.decoder.JSONDecodeError: + logger.info( + "Decoding %s failed. Probably because of comments in the json file" % json_path) + orig_dict = {} + except FileNotFoundError: + orig_dict = {} + orig_dict.update(update_dict) + with open(json_path, 'w') as f: + json.dump(orig_dict, f, indent=4) + + def __vscode_settings_cmake(self, settings_dict): + """Add cmake specific settings to settings.json. + + Note: most settings are passed to the cmake preset. + """ + if self.build_tool != BuildTool.CMAKE: + return + settings_dict["cmake.configureOnOpen"] = False + settings_dict["cmake.sourceDirectory"] = self.real_srctree + + def __gen_meson_wrapper(self): + """Generate a wrapper script to call meson with cross environment""" + bb.utils.mkdirhier(self.temp_dir) + meson_wrapper = os.path.join(self.temp_dir, 'meson') + meson_real = os.path.join( + self.recipe_sysroot_native, 'usr', 'bin', 'meson.real') + with open(meson_wrapper, 'w') as mwrap: + mwrap.write("#!/bin/sh" + os.linesep) + for var, val in self.__exported_vars.items(): + mwrap.write('export %s="%s"' % (var, val) + os.linesep) + mwrap.write("unset CC CXX CPP LD AR NM STRIP" + os.linesep) + private_temp = os.path.join(self.b, "meson-private", "tmp") + mwrap.write('mkdir -p "%s"' % private_temp + os.linesep) + mwrap.write('export TMPDIR="%s"' % private_temp + os.linesep) + mwrap.write('exec "%s" "$@"' % meson_real + os.linesep) + st = os.stat(meson_wrapper) + os.chmod(meson_wrapper, st.st_mode | stat.S_IEXEC) + self.meson_wrapper = meson_wrapper + self.cmd_compile = meson_wrapper + " compile -C " + self.b + + def __vscode_settings_meson(self, settings_dict): + if self.build_tool != BuildTool.MESON: + return + settings_dict["mesonbuild.mesonPath"] = self.meson_wrapper + + confopts = self.mesonopts.split() + confopts += self.meson_cross_file.split() + confopts += self.extra_oemeson.split() + settings_dict["mesonbuild.configureOptions"] = confopts + settings_dict["mesonbuild.buildFolder"] = self.b + + def vscode_settings(self): + files_excludes = { + "**/.git/**": True, + "**/oe-local-files/**": True, + "**/oe-logs/**": True, + "**/oe-workdir/**": True, + "**/source-date-epoch/**": True + } + python_exclude = [ + "**/.git/**", + "**/oe-local-files/**", + "**/oe-logs/**", + "**/oe-workdir/**", + "**/source-date-epoch/**" + ] + settings_dict = { + "files.watcherExclude": files_excludes, + "files.exclude": files_excludes, + "python.analysis.exclude": python_exclude + } + self.__vscode_settings_cmake(settings_dict) + self.__vscode_settings_meson(settings_dict) + + settings_file = 'settings.json' + RecipeModified.update_json_file( + self.dot_code_dir, settings_file, settings_dict) + + def __vscode_extensions_cmake(self, recommendations): + if self.build_tool != BuildTool.CMAKE: + return + recommendations += [ + "twxs.cmake", + "ms-vscode.cmake-tools", + "ms-vscode.cpptools", + "ms-vscode.cpptools-extension-pack", + "ms-vscode.cpptools-themes" + ] + + def __vscode_extensions_meson(self, recommendations): + if self.build_tool != BuildTool.MESON: + return + recommendations += [ + 'mesonbuild.mesonbuild' + ] + + def vscode_extensions(self): + recommendations = [ + "EditorConfig.EditorConfig" + ] + self.__vscode_extensions_cmake(recommendations) + self.__vscode_extensions_meson(recommendations) + extensions_file = 'extensions.json' + RecipeModified.update_json_file( + self.dot_code_dir, extensions_file, {"recommendations": recommendations}) + + def which(self, executable): + bin_path = shutil.which(executable, path=self.path) + if not bin_path: + raise DevtoolError( + 'Cannot find %s. Probably the recipe %s is not built yet.' % (executable, self.bpn)) + return bin_path + + @staticmethod + def vscode_intelli_sense_mode(compiler): + unknown = False + linux = '' + if 'linux' in compiler: + linux = "linux-" + + cxx = 'unknown-' + if 'g++' in compiler: + cxx = 'gcc-' + elif 'clang' in compiler: + cxx = 'clang-' + else: + unknown = True + + arch = 'unknown' + if 'aarch64' in compiler: + arch = 'arm64' + elif 'arm' in compiler: + arch = 'arm' + elif 'x86_64' in compiler: + arch = 'x64' + elif 'i386' in compiler: + arch = 'x86' + else: + unknown = True + + mode = linux + cxx + arch + if unknown: + logger.warn( + "Cannot guess intelliSenseMode for compiler %s (%s)" % (compiler, mode)) + return '${default}' + return mode + + def vscode_c_cpp_properties(self): + properties_dict = { + "name": "Yocto " + self.package_arch, + } + if self.build_tool == BuildTool.CMAKE: + properties_dict["configurationProvider"] = "ms-vscode.cmake-tools" + elif self.build_tool == BuildTool.MESON: + properties_dict["configurationProvider"] = "mesonbuild.mesonbuild" + + properties_dicts = { + "configurations": [ + properties_dict + ], + "version": 4 + } + prop_file = 'c_cpp_properties.json' + dot_code_dir = os.path.join(self.srctree, '.vscode') + RecipeModified.update_json_file( + dot_code_dir, prop_file, properties_dicts) + + def vscode_launch_cppdbg(self, image, gdb_cross, binary): + gdb_cross.setup_gdbserver_config(binary, self.temp_dir) + pretty_id = gdb_cross.get_gdbserver_pretty_id(binary) + gdbserver_port = gdb_cross.get_gdbserver_port(binary) + + launch_config = { + "name": pretty_id, + "type": "cppdbg", + "request": "launch", + "program": os.path.join(self.d, binary.lstrip('/')), + "stopAtEntry": True, + "cwd": "${workspaceFolder}", + "environment": [], + "externalConsole": False, + "MIMode": "gdb", + "preLaunchTask": pretty_id, + "miDebuggerPath": gdb_cross.gdb, + "miDebuggerServerAddress": "%s:%d" % (gdb_cross.host, gdbserver_port) + } + + # Search for header files in recipe-sysroot. + src_file_map = { + "/usr/include": os.path.join(self.recipe_sysroot, "usr", "include") + } + # First of all search for not stripped binaries in the image folder. + # These binaries are copied (and optionally stripped) by deploy-target + setup_commands = [ + { + "description": "sysroot", + "text": "set sysroot " + self.d + } + ] + + if image.rootfs_dbg: + launch_config['additionalSOLibSearchPath'] = self.solib_search_path_str( + image) + src_file_map["/usr/src/debug"] = os.path.join( + image.rootfs_dbg, "usr", "src", "debug") + else: + logger.warning( + "Cannot setup debug symbols configuration for GDB. IMAGE_GEN_DEBUGFS is not enabled.") + + launch_config['sourceFileMap'] = src_file_map + launch_config['setupCommands'] = setup_commands + return launch_config + + @staticmethod + def is_elf_file(file_path): + with open(file_path, "rb") as f: + data = f.read(4) + if data == b'\x7fELF': + return True + return False + + def find_installed_binaries(self): + """find all executable elf files in the image directory""" + binaries = [] + d_len = len(self.d) + re_so = re.compile('.*\.so[.0-9]*$') + for root, _, files in os.walk(self.d, followlinks=False): + for file in files: + if os.path.islink(file): + continue + if re_so.match(file): + continue + abs_name = os.path.join(root, file) + if os.access(abs_name, os.X_OK) and RecipeModified.is_elf_file(abs_name): + binaries.append(abs_name[d_len:]) + return binaries + + def vscode_launch(self, image, gdb_cross): + binaries = self.find_installed_binaries() + configurations = [self.vscode_launch_cppdbg( + image, gdb_cross, binary) for binary in binaries] + launch_dict = { + "version": "0.2.0", + "configurations": configurations + } + launch_file = 'launch.json' + RecipeModified.update_json_file( + self.dot_code_dir, launch_file, launch_dict) + + @staticmethod + def get_unique_gdbinit_name(binary): + return 'gdbinit' + binary.replace(os.sep, '-') + + def none_launch(self, image, gdb_cross): + """generate a gdbinit file per executable""" + binaries = self.find_installed_binaries() + for binary in binaries: + gdb_cross.setup_gdbserver_config(binary, self.temp_dir) + gdbserver_port = str(gdb_cross.get_gdbserver_port(binary)) + if gdb_cross.gdbserver_multi: + target_help = '# gdbserver --multi :' + gdbserver_port + remote_cmd = 'target extended-remote ' + else: + target_help = '# gdbserver :' + gdbserver_port + ' ' + binary + remote_cmd = 'target remote ' + gdbinit_path = os.path.join( + self.real_srctree, RecipeModified.get_unique_gdbinit_name(binary)) + + gdbinit_lines = ['# This file is generated by devtool ide'] + gdbinit_lines.append('# On the remote target:') + gdbinit_lines.append(target_help) + gdbinit_lines.append('# On the build machine:') + gdbinit_lines.append('# cd ' + self.real_srctree) + gdbinit_lines.append( + '# ' + gdb_cross.gdb + ' -ix ' + gdbinit_path) + + gdbinit_lines.append('set sysroot ' + self.d) + gdbinit_lines.append('set substitute-path "/usr/include" "' + + os.path.join(self.recipe_sysroot, 'usr', 'include') + '"') + if image.debuginfod: + gdbinit_lines.append('set debuginfod enabled on') + else: + gdbinit_lines.append('set debuginfod enabled off') + if image.rootfs_dbg: + gdbinit_lines.append( + 'set solib-search-path "' + self.solib_search_path_str(image) + '"') + gdbinit_lines.append('set substitute-path "/usr/src/debug" "' + os.path.join( + image.rootfs_dbg, 'usr', 'src', 'debug') + '"') + gdbinit_lines.append( + remote_cmd + gdb_cross.host + ':' + gdbserver_port) + gdbinit_lines.append('set remote exec-file ' + binary) + gdbinit_lines.append('run ' + os.path.join(self.d, binary)) + + with open(gdbinit_path, 'w') as gdbinit_file: + gdbinit_file.write('\n'.join(gdbinit_lines)) + + def gen_fakeroot_install_script(self): + """Run the run.do_install script from bitbake under pseudo so that it picks up the appropriate file permissions""" + cmd_lines = ['#!/bin/sh'] + # Ensure the do compile step gets always executed without pseuso before do install + if self.cmd_compile: + cmd_compile = "( cd %s && %s)" % ( + self.real_srctree, self.cmd_compile) + cmd_lines.append(cmd_compile) + if not os.access(self.fakerootcmd, os.X_OK): + raise DevtoolError( + "pseudo executable %s could not be found" % self.fakerootcmd) + run_do_install = os.path.join(self.workdir, 'temp', 'run.do_install') + + if not os.access(run_do_install, os.X_OK): + raise DevtoolError( + "run script does not exists: %s" % run_do_install) + + # Set up the appropriate environment + newenv = dict(os.environ) + for varvalue in self.fakerootenv.split(): + if '=' in varvalue: + splitval = varvalue.split('=', 1) + newenv[splitval[0]] = splitval[1] + + # Cleanup TMPDIR before calling do_install independently from bitbake + # This is anyway outdated after do_install has been executed. + # But maybe there should be a cleaner solution here. + rm_in_workdir = ' '.join([os.path.join(self.workdir, d) for d in [ + "package", "packages-split", "pkgdata", "sstate-install-package", "debugsources.list", "*.spec"]]) + install_cmd = '%s /bin/sh -c "rm -rf %s/* %s && %s"' % ( + self.fakerootcmd, self.d, rm_in_workdir, run_do_install) + for var, val in newenv.items(): + cmd_lines.append('export %s="%s"' % (var, val)) + cmd_lines.append(install_cmd) + return self.write_script(cmd_lines, 'bb_run_do_install') + + def write_script(self, cmd_lines, script_name): + bb.utils.mkdirhier(self.temp_dir) + script_file = os.path.join(self.temp_dir, script_name) + with open(script_file, 'w') as script_f: + script_f.write(os.linesep.join(cmd_lines)) + st = os.stat(script_file) + os.chmod(script_file, st.st_mode | stat.S_IEXEC) + return script_file + + def vscode_tasks(self, args, gdb_cross): + run_do_install = self.gen_fakeroot_install_script() + pythonpath_new = os.path.realpath( + os.path.join(self.bitbakepath, '../lib')) + try: + pythonpath = os.environ['PYTONPATH'].split(':') + if pythonpath_new not in pythonpath: + pythonpath = pythonpath_new + ":" + pythonpath + except KeyError: + pythonpath = pythonpath_new + install_label = "do_install %s" % self.bpn + deploy_label = "devtool deploy-target %s" % self.bpn + bb_env = { + # "BBPATH": self.bbpath, + "PYTHONPATH": pythonpath, + "BUILDDIR": self.topdir, + # Use e.g. Python from host, not python-native + "PATH": os.environ['PATH'], + "BB_ENV_PASSTHROUGH_ADDITIONS": self.bb_env_passthrough_additions + } + bb_options = { + "cwd": self.topdir + } + tasks_dict = { + "version": "2.0.0", + "tasks": [ + { + "label": install_label, + "type": "shell", + "command": run_do_install, + "problemMatcher": [] + }, + { + "label": deploy_label, + "type": "shell", + "command": "devtool", # TODO: Generate a self contained script which does what devtool target-deploy does but without connection to the bitbake server + "args": ["--bbpath", self.bbpath, "deploy-target", self.bpn, args.target, "--strip", "--no-host-check"], + "linux": { + "options": { + "env": bb_env + }}, + "options": bb_options, + "problemMatcher": [] + }, + { + "label": "install && deploy-target %s" % self.bpn, + "dependsOrder": "sequence", + "dependsOn": [ + install_label, + deploy_label + ], + "problemMatcher": [], + "group": { + "kind": "build", + "isDefault": True + } + } + ] + } + for pretty_id, start_script in gdb_cross.get_gdbserver_start_scripts(): + tasks_dict['tasks'].append( + { + "label": pretty_id, + "type": "shell", + "isBackground": True, + "command": start_script, + "problemMatcher": [ + { + "pattern": [ + { + "regexp": ".", + "file": 1, + "location": 2, + "message": 3 + } + ], + "background": { + "activeOnStart": True, + "beginsPattern": ".", + "endsPattern": ".", + } + } + ] + }) + tasks_file = 'tasks.json' + RecipeModified.update_json_file( + self.dot_code_dir, tasks_file, tasks_dict) + + def setup_ide(self, args, image, gdb_cross): + if self.build_tool == BuildTool.CMAKE: + self.__cmake_preset() + if self.build_tool == BuildTool.MESON: + self.__gen_meson_wrapper() + + if args.ide == 'code': + self.vscode_settings() + self.vscode_extensions() + self.vscode_c_cpp_properties() + if args.target: + self.vscode_launch(image, gdb_cross) + self.vscode_tasks(args, gdb_cross) + if args.ide == 'none' and self.build_tool == BuildTool.CMAKE: + self.none_launch(image, gdb_cross) + + +def ide_setup(args, config, basepath, workspace): + bootstap_tasks = [] + + tinfoil = setup_tinfoil(config_only=False, basepath=basepath) + try: + # Provide a rootfs and the corresponding debug symbols via rootfs-dbg or debuginfod + image_config = RecipeImage(args.image) + image_config.initialize(config, tinfoil) + bootstap_tasks += image_config.bootstrap_tasks + + target_device = TargetDevice(args) + + sdk_env = None + gdb_cross = None + if args.recipename in SHARED_SYSROOT_RECIPES: + ide_support = RecipeMetaIdeSupport() + ide_support.initialize(config, tinfoil) + bootstap_tasks += ide_support.bootstrap_tasks + + build_sysroots = RecipeBuildSysroots() + build_sysroots.initialize(config, tinfoil) + bootstap_tasks += build_sysroots.bootstrap_tasks + + sdk_env = SharedSysrootsEnv() + sdk_env.initialize(ide_support, build_sysroots) + else: + sdk_env = RecipeModified(args.recipename) + sdk_env.initialize(config, workspace, tinfoil) + bootstap_tasks += sdk_env.bootstrap_tasks + + gdb_multi_mode = not bool(args.ide == 'code') + gdb_cross = RecipeGdbCross( + args, sdk_env.target_arch, target_device, gdb_multi_mode) + gdb_cross.initialize(config, workspace, tinfoil) + bootstap_tasks += gdb_cross.bootstrap_tasks + finally: + tinfoil.shutdown() + + shared_sysroot = bool(args.recipename in SHARED_SYSROOT_RECIPES) + if not shared_sysroot: + sdk_env.debug_build_config(args) + + if not args.skip_bitbake: + bb_cmd = 'bitbake ' + if args.bitbake_k: + bb_cmd += "-k " + bb_cmd += ' '.join(bootstap_tasks) + exec_build_env_command(config.init_path, basepath, bb_cmd, watch=True) + + if shared_sysroot: + sdk_env.setup_ide(args) + else: + sdk_env.setup_ide(args, image_config, gdb_cross) + + +def get_default_ide(): + for an_ide in SUPPORTED_IDES[:-1]: + if shutil.which(an_ide): + return an_ide + return SUPPORTED_IDES[-1:] + + +def register_commands(subparsers, context): + """Register devtool subcommands from this plugin""" + parser_ide = subparsers.add_parser('ide', help='Setup the IDE (VSCode)', + description='Configure the IDE to work with the source code of a recipe.', + group='working', order=50, formatter_class=RawTextHelpFormatter) + parser_ide.add_argument( + 'recipename', help='Generate a IDE configuration in the workspace of the given recipe. ' + 'By default the workspace is configured to use the recipe sysroot prepared by devtool modify. ' + 'For some special recipes the generated configuration referes to the shared sysroots ' + 'provided by meta-ide-setup and build-sysroots recipes. ' + 'The following recipes use a shared sysroot: %s' % SHARED_SYSROOT_RECIPES) + parser_ide.add_argument( + 'image', help='The image running on the target device. This is required for remote debugging.' + 'It is important to deploy the image built by this command to the target device because ' + 'otherwise the debug symbols used on the build machine (rootfs-dbg) are probably out of sync ' + 'with the binaries executed on the target device.') + parser_ide.add_argument( + '-i', '--ide', choices=SUPPORTED_IDES, default=get_default_ide(), + help='Setup the configuration for this IDE') + parser_ide.add_argument( + '-t', '--target', default='root@192.168.7.2', + help='Live target machine running an ssh server: user@hostname.') + parser_ide.add_argument( + '-G', '--gdbserver-port-start', default="1234", help='port where gdbserver is listening.') + parser_ide.add_argument( + '-c', '--no-host-check', help='Disable ssh host key checking', action='store_true') + parser_ide.add_argument( + '-e', '--ssh-exec', help='Executable to use in place of ssh') + parser_ide.add_argument( + '-P', '--port', help='Specify ssh port to use for connection to the target') + parser_ide.add_argument( + '-I', '--key', help='Specify ssh private key for connection to the target') + parser_ide.add_argument( + '--skip-bitbake', help='Generate IDE configuration but skip calling bibtake to update the SDK.', action='store_true') + parser_ide.add_argument( + '-k', '--bitbake-k', help='Pass -k parameter to bitbake', action='store_true') + parser_ide.add_argument( + '--debug-build-config', help='Use debug build flags, for example set CMAKE_BUILD_TYPE=Debug', action='store_true') + parser_ide.set_defaults(func=ide_setup) + + # TODO: Better support for multiple recipes. E.g. a list of recipes with auto-detection for the image recipe or all modified recipes From patchwork Tue Sep 5 16:51:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 306FDCA0FFF for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.web10.27272.1693933177169931341 for ; Tue, 05 Sep 2023 09:59:37 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=le+hygce; spf=pass (domain: gmail.com, ip: 209.85.128.48, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-3fbd33a57b6so25750025e9.2 for ; Tue, 05 Sep 2023 09:59:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933175; x=1694537975; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VaWuuIUtmaan9OQ7nZXD7CNTUSo25dmTfn2Y4hW5+Sg=; b=le+hygceUVI26cHTAzULqENEjVX2EuayqbNE+pdvUwAiTCahgS3kh1H4IcFvr3MhGR Dv7d2roSk1USIpzIFGd9g1hu0qtxcJy9FlEuuJHNTi/2rlh1GaD+MYNYtjXCeZl4yb+b UdeYDfK7v+pgHVPtfY+cHzkT2InjsLVGR2+6yQu0vE7NyK+1fGFNwDmvgmot0ypww3PO +q38QRtKHVdivH9HBKB1E/1a9OOnbgPuhH40duw5RKqUGppjDC8M/QA0KOiivLyzcpdr c/8rSx1PkzB2T5j1OIW6c/22SFCz7A7rJRs5xzXkKmKcWiO05h8Gx4++h11bwRazv5xx WDkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933175; x=1694537975; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VaWuuIUtmaan9OQ7nZXD7CNTUSo25dmTfn2Y4hW5+Sg=; b=Gw+5/Ug7VJ70+rTYw/pe8V/7ARUPPDmo1ootqWnIGInLKE857V661wJVSwxOvowH3o FE4p8TP6/9E8eiWk/1367nNzLO1r275lC9DZjCqEHRDQiqb4RCGbfedenPK+StG2pfy8 7UFaJO3vgzBrDTypkuvoELbFTiuZ9GIiJxBeGCAnSbMROt0czGmbwcYfoDaMiacOolUY cPONnZssNTebMnPfd8kj5bdLNnc/LyFwOQr6HcFDUhsO8lObjYW+Fz6T1lXB4YjwTQ+n /MWdiOZ+ndksre70JGb89+LD9koozuMQWrSjeONcsMnhZQGoDBRe9NiuJWH4A+e61a/K 1kCA== X-Gm-Message-State: AOJu0YwMnFNDZV0Xc0yeDc9rNKo/y4aNVByomluFfwuR4kcFHdhrqplq wI0x2agQSgUiXFUCB/KmBiuNmhyYs8Q= X-Google-Smtp-Source: AGHT+IHL4cuoo1U4coqC3IfGtFw3fqqOX+4hFirCk81+qGBX1EOkV1YlWg7zwzkPs2G1LKhxom5hDw== X-Received: by 2002:a05:6000:12cc:b0:318:8b5:4159 with SMTP id l12-20020a05600012cc00b0031808b54159mr337580wrx.24.1693933175355; Tue, 05 Sep 2023 09:59:35 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:35 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 06/14] tests: add a C++ example recipe Date: Tue, 5 Sep 2023 18:51:15 +0200 Message-ID: <20230905165913.2560907-7-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187238 This simple C++ project supports compilation with cmake and with meson. It's supposed to be used with oe-selftest for the devtool ide plugin. Signed-off-by: Adrian Freihofer --- meta-selftest/recipes-test/cpp/.gitignore | 1 + .../recipes-test/cpp/cmake-example.bb | 17 ++++++ .../recipes-test/cpp/cmake-example/run-ptest | 10 ++++ .../recipes-test/cpp/cpp-example.inc | 24 ++++++++ .../recipes-test/cpp/files/CMakeLists.txt | 60 +++++++++++++++++++ .../cpp/files/cpp-example-lib.cpp | 17 ++++++ .../cpp/files/cpp-example-lib.hpp | 16 +++++ .../recipes-test/cpp/files/cpp-example.cpp | 16 +++++ .../recipes-test/cpp/files/meson.build | 34 +++++++++++ .../cpp/files/test-cpp-example.cpp | 19 ++++++ .../recipes-test/cpp/meson-example.bb | 17 ++++++ .../recipes-test/cpp/meson-example/run-ptest | 10 ++++ 12 files changed, 241 insertions(+) create mode 100644 meta-selftest/recipes-test/cpp/.gitignore create mode 100644 meta-selftest/recipes-test/cpp/cmake-example.bb create mode 100644 meta-selftest/recipes-test/cpp/cmake-example/run-ptest create mode 100644 meta-selftest/recipes-test/cpp/cpp-example.inc create mode 100644 meta-selftest/recipes-test/cpp/files/CMakeLists.txt create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example.cpp create mode 100644 meta-selftest/recipes-test/cpp/files/meson.build create mode 100644 meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp create mode 100644 meta-selftest/recipes-test/cpp/meson-example.bb create mode 100644 meta-selftest/recipes-test/cpp/meson-example/run-ptest diff --git a/meta-selftest/recipes-test/cpp/.gitignore b/meta-selftest/recipes-test/cpp/.gitignore new file mode 100644 index 00000000000..30d388a12b7 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/.gitignore @@ -0,0 +1 @@ +build* \ No newline at end of file diff --git a/meta-selftest/recipes-test/cpp/cmake-example.bb b/meta-selftest/recipes-test/cpp/cmake-example.bb new file mode 100644 index 00000000000..96d543180b4 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/cmake-example.bb @@ -0,0 +1,17 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +SUMMARY = "A C++ example compiled with cmake." + +inherit cmake + +require cpp-example.inc + +SRC_URI += "\ + file://CMakeLists.txt \ +" + +FILES:${PN}-ptest += "${bindir}/test-cmake-example" diff --git a/meta-selftest/recipes-test/cpp/cmake-example/run-ptest b/meta-selftest/recipes-test/cpp/cmake-example/run-ptest new file mode 100644 index 00000000000..94b620a1984 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/cmake-example/run-ptest @@ -0,0 +1,10 @@ +#!/bin/sh +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +test-cmake-example + +# Note: run-ptests exits with exit value from test-cmake-example diff --git a/meta-selftest/recipes-test/cpp/cpp-example.inc b/meta-selftest/recipes-test/cpp/cpp-example.inc new file mode 100644 index 00000000000..39c61cf4ceb --- /dev/null +++ b/meta-selftest/recipes-test/cpp/cpp-example.inc @@ -0,0 +1,24 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" + +inherit ptest + +DEPENDS += "json-c" + +PV = "1.0" + +S = "${WORKDIR}" + +SRC_URI = "\ + file://cpp-example.cpp \ + file://cpp-example-lib.hpp \ + file://cpp-example-lib.cpp \ + file://test-cpp-example.cpp \ + file://run-ptest \ +" diff --git a/meta-selftest/recipes-test/cpp/files/CMakeLists.txt b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt new file mode 100644 index 00000000000..839aa59b5e3 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt @@ -0,0 +1,60 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22) + +project(cmake-example + VERSION 1.0.0 + LANGUAGES CXX +) + +option(BUILD_SHARED_LIBS "Build using shared libraries" ON) + +set(CMAKE_CXX_STANDARD 17) +set(CMAKE_CXX_STANDARD_REQUIRED On) +set(CMAKE_CXX_EXTENSIONS Off) + +include(GNUInstallDirs) + +# Find json-c +# find_package(PkgConfig REQUIRED) +# pkg_check_modules(JSONC REQUIRED json-c) +find_package(json-c) + +# A simple library linking json-c library found by pkgconfig +add_library(cmake-example-lib cpp-example-lib.cpp cpp-example-lib.hpp) +set_target_properties(cmake-example-lib PROPERTIES + VERSION ${PROJECT_VERSION} + SOVERSION ${PROJECT_VERSION_MAJOR} +) +target_link_libraries(cmake-example-lib PRIVATE json-c::json-c) +# target_link_libraries(cmake-example-lib ${JSONC_LIBRARIES}) +# target_include_directories(cmake-example-lib PUBLIC ${JSONC_INCLUDE_DIRS}) +# target_compile_options(cmake-example-lib PUBLIC ${JSONC_CFLAGS_OTHER}) +install(TARGETS cmake-example-lib + INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} + ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} + LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} +) + +# A simple executable linking the library +add_executable(cmake-example cpp-example.cpp) +target_link_libraries(cmake-example PRIVATE cmake-example-lib) + +install(TARGETS cmake-example + RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} +) + +# A simple test executable for testing the library +add_executable(test-cmake-example test-cpp-example.cpp) +target_link_libraries(test-cmake-example PRIVATE cmake-example-lib) + +install(TARGETS test-cmake-example + RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} +) + +include(CTest) +add_test(NAME test-cmake-example COMMAND test-cmake-example) diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp new file mode 100644 index 00000000000..29fd10fb7e1 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp @@ -0,0 +1,17 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#include +#include +#include "cpp-example-lib.hpp" + +const std::string& CppExample::get_string() { + return test_string; +} + +const char* CppExample::get_json_c_version() { + return json_c_version(); +} diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp new file mode 100644 index 00000000000..8a87e6b510a --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp @@ -0,0 +1,16 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#pragma once + +#include + +struct CppExample { + inline static const std::string test_string = "cpp-example-lib Magic: 123456789"; + + const std::string& get_string(); + const char* get_json_c_version(); +}; diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example.cpp b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp new file mode 100644 index 00000000000..de7f2106dec --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp @@ -0,0 +1,16 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#include "cpp-example-lib.hpp" + +#include + +int main() { + auto cpp_example = CppExample(); + std::cout << "C++ example linking " << cpp_example.get_string() << std::endl; + std::cout << "Linking json-c version " << cpp_example.get_json_c_version() << std::endl; + return 0; +} diff --git a/meta-selftest/recipes-test/cpp/files/meson.build b/meta-selftest/recipes-test/cpp/files/meson.build new file mode 100644 index 00000000000..7753da69721 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/meson.build @@ -0,0 +1,34 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +project('meson-example', 'cpp', + version: '1.0.0', + default_options: ['cpp_std=c++17'] + ) + + +jsoncdep = dependency('json-c') + +mesonexlib = shared_library('mesonexlib', + 'cpp-example-lib.cpp', 'cpp-example-lib.hpp', + version: meson.project_version(), + soversion: meson.project_version().split('.')[0], + dependencies : jsoncdep, + install : true + ) + +executable('mesonex', + 'cpp-example.cpp', + link_with : mesonexlib, + install : true + ) + +test_mesonex = executable('test-mesonex', + 'test-cpp-example.cpp', + link_with : mesonexlib, + install : true +) +test('meson example test', test_mesonex) diff --git a/meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp b/meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp new file mode 100644 index 00000000000..ca72e11a9c4 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp @@ -0,0 +1,19 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#include "cpp-example-lib.hpp" + +#include + +int main() { + auto cpp_example = CppExample(); + auto ret_string = cpp_example.get_string(); + if(0 == ret_string.compare(CppExample::test_string)) { + std::cout << "PASS: " << ret_string << " = " << CppExample::test_string << std::endl; + } else { + std::cout << "FAIL: " << ret_string << " != " << CppExample::test_string << std::endl; + } +} diff --git a/meta-selftest/recipes-test/cpp/meson-example.bb b/meta-selftest/recipes-test/cpp/meson-example.bb new file mode 100644 index 00000000000..f15dbb50731 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/meson-example.bb @@ -0,0 +1,17 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +SUMMARY = "A C++ example compiled with meson." + +inherit pkgconfig meson + +require cpp-example.inc + +SRC_URI += "\ + file://meson.build \ +" + +FILES:${PN}-ptest += "${bindir}/test-mesonex" diff --git a/meta-selftest/recipes-test/cpp/meson-example/run-ptest b/meta-selftest/recipes-test/cpp/meson-example/run-ptest new file mode 100644 index 00000000000..b1804f00961 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/meson-example/run-ptest @@ -0,0 +1,10 @@ +#!/bin/sh +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +test-mesonex + +# Note: run-ptests exits with exit value from test-mesonex From patchwork Tue Sep 5 16:51:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18DA7CA0FFE for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by mx.groups.io with SMTP id smtpd.web10.27273.1693933177951740715 for ; Tue, 05 Sep 2023 09:59:38 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=EKolGiDz; spf=pass (domain: gmail.com, ip: 209.85.128.51, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-40078c4855fso27189035e9.3 for ; Tue, 05 Sep 2023 09:59:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933176; x=1694537976; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r/JU2AOFiMtjJt9nxjy8Xjf4md5X6Dyo8Xa9UuZQhDM=; b=EKolGiDzhzcExF6aXSpNUyzy9sBBbdScUhJPt0Z63xYVYtiIYbGCq8E9kwobXFU5Jh 4UjlZQuPYKRxwJOa1wB/pTuHjXcxBkPUdgikv8wwvpt0iRS462ZPELLqbZzx5OUezDqq xxNi+EYgSvPtCemDkFAymtME4aqnWZAwGjJ0zL94tTIXUd/HliPgEjZh+2kN4T1ypahB H9BxYENGJJFNVIQ5gELPyViqFwHrqrGAB9IOceQY3TeDUFBm5wH3fQURwmN1cSO0XY97 3wk2TgSBJCv5W65wwnXxYQtjIWEO/tbsV7WA09/sdruK226yQfIpUnjidXJ/WbvXF+2z kWDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933176; x=1694537976; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r/JU2AOFiMtjJt9nxjy8Xjf4md5X6Dyo8Xa9UuZQhDM=; b=Y4NU9Tg/EXcEi5vjv+dTAV8VIB4GkMJ3jcA56BiH6C33HKzuSsfX7dSH6B/UeYaCXP xxrMV4BypbImDlNJmyBEiXO82uJfepxt3lDjAd2c3HGLDwLe/j0Biq3QHDoxviRjULJ7 rQ7Mm557SAx3PQf8Ow3jB5XWdMp/jR5+82tBvfnlDRASxGuzJ3WAw2mRXzpRRxs8Nsmt sqEpVBPgS6MbATG+4E28EZKEgy1MEikvrjZS/mpL427FprVD5qKYgbzY3XH5/OsW6F5M 733fygzAA7ZTX6LDSJMLm/Tb4Ck6ufdNYr1/sjh7rijrG/jDCd1ax2yod/kt++J6ECFk mfKw== X-Gm-Message-State: AOJu0YzyhtFfmMQqwwmHkXCj1TAXS9oQA3qE1pn/BOMurlLSMibh4NUv EqFwHpV7OJ4hxsR15VOU67HB1g5OmMw= X-Google-Smtp-Source: AGHT+IGqwHNvNArwaym7AkZljj6EAGcgPUm9wAZxHwVN3kMNv266RsJUwBGALhQDf0Ds9/pWo+ywrw== X-Received: by 2002:a05:600c:ac1:b0:401:b3a5:ec03 with SMTP id c1-20020a05600c0ac100b00401b3a5ec03mr288859wmr.1.1693933176096; Tue, 05 Sep 2023 09:59:36 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:35 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 07/14] cmake-example: workaround for pseudo breakeage Date: Tue, 5 Sep 2023 18:51:16 +0200 Message-ID: <20230905165913.2560907-8-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187239 Signed-off-by: Adrian Freihofer --- meta-selftest/recipes-test/cpp/cmake-example.bb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta-selftest/recipes-test/cpp/cmake-example.bb b/meta-selftest/recipes-test/cpp/cmake-example.bb index 96d543180b4..fbf1f266721 100644 --- a/meta-selftest/recipes-test/cpp/cmake-example.bb +++ b/meta-selftest/recipes-test/cpp/cmake-example.bb @@ -15,3 +15,9 @@ SRC_URI += "\ " FILES:${PN}-ptest += "${bindir}/test-cmake-example" + +# This is a workaround for packages with sources in-tree +# Without this bitbake ... does not work after running the install script. +# newenv['PSEUDO_IGNORE_PATHS'] = newenv['PSEUDO_IGNORE_PATHS'] + "" +# path mismatch [3 links]: ino 37529096 db '/home/adrian/projects/oss/meta-yocto-upstream/projects/poky-oe-glibc-sd/tmp/work/cortexa57-poky-linux/cmake-example/1.0-r0/package/usr/src/debug/cmake-example/1.0-r0/oe-local-files/cmake-example-lib.cpp' req '/home/adrian/projects/oss/meta-yocto-upstream/projects/poky-oe-glibc-sd/workspace/sources/cmake-example/oe-local-files/cmake-example-lib.cpp'. +PACKAGE_DEBUG_SPLIT_STYLE = "debug-without-src" From patchwork Tue Sep 5 16:51:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18768CA0FFB for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web10.27274.1693933178688884181 for ; Tue, 05 Sep 2023 09:59:39 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=OQnSziAN; spf=pass (domain: gmail.com, ip: 209.85.221.47, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-313e742a787so23879f8f.1 for ; Tue, 05 Sep 2023 09:59:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933177; x=1694537977; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tmsQxkNVWaDgGBhR/wx42mDBkC+iCPyEtuWdQmFXKe0=; b=OQnSziANLmFlc3AKMQxuu7PSmC69Fre/OK7e4mNEYhZa5wSuYjrp8KlbgqEr9jXnc8 YHdTiAYTRZHJ3psjXa9LkTo4P7ExcQTBlhR4OtU5Y5qHNN/KMjRFUTDz9XbkMhh7YLgY gL7MeH4Ktgki2wko0DOb1WOnbpP07g7/EvQFP+qssNerEvcK4jbjZkCjq8RNZjJZwnfN qWM3PjTR50aC3u961on3ThrjAowyMoNSBLP1tVl5iRGVS0FcBtdND7ro8VNywhojnJI4 UB8COAVRCFWx/DJD+Xemc7vrLijyWfU9YD3k7jNH5IL5bSk8yR+jvTfVN5QEea+xGO9/ GWSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933177; x=1694537977; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tmsQxkNVWaDgGBhR/wx42mDBkC+iCPyEtuWdQmFXKe0=; b=MGLnDQWJwXBbgEtJEWk/lddt33nY7ldmDlEnXOkh3VI/bkGQc5eJ9/p2luoMZD4rk7 fvppRfLVVHOxnF9wcywYKckLQMla3YrJQ23pKzo0jKovMNB62go7h3PVLqyZmMnTLx15 kL7QzOA2Jp/SFEkzjV6Q3wdmFrfoHI25V7VCBkQUwQS7iB5meUPO1qUgkkBIX/AnedTG NmnwrWgyle2xYWr+xteOkmDhF85uwW0blWLcyltpaufSp1ePt63Oe1NEnLTenAKjNckX 88RbPVALiT+pkD9Am/686CEDaJBE90bKwJGR0X8SVAUVSKK+u6J8oITOLIAWVCWBTJUB 4+bw== X-Gm-Message-State: AOJu0YyhwdU4iaVOu9G2BMKELuj5KQgF2D5xaZEnxn2nbXUr2pFKSeWR /O+XLn6qZSBODIRjrDzcTvp9EFHAJls= X-Google-Smtp-Source: AGHT+IF/xRQIyOeDwJZQgKRI4dXC+huDOBf5f5JpRSJLFWjLVvRr9VGG8x+H7hcfJbHkwRTDjQwOVg== X-Received: by 2002:adf:fb10:0:b0:313:e2e3:d431 with SMTP id c16-20020adffb10000000b00313e2e3d431mr318049wrr.12.1693933176875; Tue, 05 Sep 2023 09:59:36 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:36 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 08/14] refactor: make multiprocess_launch callable without d Date: Tue, 5 Sep 2023 18:51:17 +0200 Message-ID: <20230905165913.2560907-9-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187240 This is a preparation for making the strip_execs function callable from devtool without going via tinfoil and a bitbake server process. Signed-off-by: Adrian Freihofer --- meta/conf/bitbake.conf | 2 +- meta/lib/oe/utils.py | 12 +++++++++--- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 0bcc8215c5d..69e854ecedc 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -975,5 +975,5 @@ BB_UNIHASH ?= "${BB_TASKHASH}" oe.sstatesig.find_sstate_manifest[vardepsexclude] = "BBEXTENDCURR BBEXTENDVARIANT OVERRIDES PACKAGE_EXTRA_ARCHS" oe.utils.get_multilib_datastore[vardepsexclude] = "DEFAULTTUNE_MULTILIB_ORIGINAL OVERRIDES" oe.path.format_display[vardepsexclude] = "TOPDIR" -oe.utils.multiprocess_launch[vardepsexclude] = "BB_NUMBER_THREADS" +oe.utils.get_bb_number_threads[vardepsexclude] = "BB_NUMBER_THREADS" oe.packagedata.emit_pkgdata[vardepsexclude] = "BB_NUMBER_THREADS" diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py index 69ca8987f3d..39042463d6b 100644 --- a/meta/lib/oe/utils.py +++ b/meta/lib/oe/utils.py @@ -264,10 +264,17 @@ def execute_pre_post_process(d, cmds): bb.note("Executing %s ..." % cmd) bb.build.exec_func(cmd, d) -# For each item in items, call the function 'target' with item as the first +def get_bb_number_threads(d): + return int(d.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) + +def multiprocess_launch(target, items, d, extraargs=None): + max_process = get_bb_number_threads(d) + return multiprocess_launch_mp(target, items, max_process, extraargs) + +# For each item in items, call the function 'target' with item as the first # argument, extraargs as the other arguments and handle any exceptions in the # parent thread -def multiprocess_launch(target, items, d, extraargs=None): +def multiprocess_launch_mp(target, items, max_process, extraargs=None): class ProcessLaunch(multiprocessing.Process): def __init__(self, *args, **kwargs): @@ -302,7 +309,6 @@ def multiprocess_launch(target, items, d, extraargs=None): self.update() return self._result - max_process = int(d.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) launched = [] errors = [] results = [] From patchwork Tue Sep 5 16:51:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06A7ECA0FF3 for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.web11.27502.1693933179405316031 for ; Tue, 05 Sep 2023 09:59:39 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=rGIB78wr; spf=pass (domain: gmail.com, ip: 209.85.128.48, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-401da71b83cso26402965e9.2 for ; Tue, 05 Sep 2023 09:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933177; x=1694537977; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7c86nxg2BMAsLSg0HXWALOut08e2w5F1zYwmYIPR1IE=; b=rGIB78wrX0jh+Ldj+vbTnwKAwACrTsUb5yeOtAH83BIzBrmzZNA8vZSAvydVhAhpgJ 8grBT4UdI6Z6aPpHOK3zITWnVbEHE8vtWQql53fy1XnSrLGa4/IIxrS5IrvHyGfzYa2e nFHGKMGzDhrNizmk3WO5egot2FKOs6wWJd1lvKgnRc+jyN5cUPsUjwXbPcSM06li0A2a tvYz1fwCPOP/OkYHa1s4DvQ+Kp/2splHPdBtwh0sqA5Gdh4I181+fAtxI/BXWjhAg0H0 SHR38DlUeOcDwyWQKIFFvrePvveRgbrm1o7Xii5b/E9bSaKO6h/6PwacOKL9POY2sZhl hQOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933177; x=1694537977; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7c86nxg2BMAsLSg0HXWALOut08e2w5F1zYwmYIPR1IE=; b=XDu56fv15SA38C87LfW367mW9dSdNWQKepAG5DrIDBKfrGnuXr6GocaOjSnpqpiwkc YKng3xISSwBdoSst14mEWqGyN0uQvv597QrcG7QIEe7LVzYK3qBxt2NH4gFug7ho+Put /5Xmyg7UKDTtyUyv0o1OjsePvxc9EM/XcsiPhP7IMetuinv51z+NBIcOux7zyBSqnKw1 Oy4PdUusUJPM9ahdpQz4l2+mVu93NIWmup0aNNIkOty30vDbwLiVdyGtZXHHzj93M35q c77bxsZ2Qkoa1ShrrU06Q4jGFbPS92a4SnV6/Iua3cedn2eisseR6bY+o2H2sGZS8Fbp yO3g== X-Gm-Message-State: AOJu0YwTj+hSrM0l9+VNWap5T05yVK5JnygcXIWE+aId6KkGfJhxEAs4 lJtlldgim06YWSs3jKwE+lBEaX2JnGw= X-Google-Smtp-Source: AGHT+IEwwHxcX0jzPN0eFXigakBGQDbUDTrtPLSiyjbZs8IZ6cn1UY2dXpszO8GuhUpiFaxRaVMUug== X-Received: by 2002:adf:e7cb:0:b0:319:8c35:37b with SMTP id e11-20020adfe7cb000000b003198c35037bmr328535wrn.7.1693933177632; Tue, 05 Sep 2023 09:59:37 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:37 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 09/14] refactor: make strip_execs callable without d Date: Tue, 5 Sep 2023 18:51:18 +0200 Message-ID: <20230905165913.2560907-10-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187241 This allows to call strip_execs function from devtool without going via tinfoil and a bitbake server process. Signed-off-by: Adrian Freihofer --- meta/classes-global/staging.bbclass | 3 ++- meta/lib/oe/package.py | 7 ++++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/meta/classes-global/staging.bbclass b/meta/classes-global/staging.bbclass index 3a300c32e7c..d229f401073 100644 --- a/meta/classes-global/staging.bbclass +++ b/meta/classes-global/staging.bbclass @@ -92,7 +92,8 @@ python sysroot_strip () { qa_already_stripped = 'already-stripped' in (d.getVar('INSANE_SKIP:' + pn) or "").split() strip_cmd = d.getVar("STRIP") - oe.package.strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, + max_process = oe.utils.get_bb_number_threads(d) + oe.package.strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, max_process, qa_already_stripped=qa_already_stripped) } diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 9d70925b9b7..1dd20f85ebd 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -114,7 +114,7 @@ def is_static_lib(path): return start == magic return False -def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripped=False): +def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, max_process, qa_already_stripped=False): """ Strip executable code (like executables, shared libraries) _in_place_ - Based on sysroot_strip in staging.bbclass @@ -122,6 +122,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp :param strip_cmd: Strip command (usually ${STRIP}) :param libdir: ${libdir} - strip .so files in this directory :param base_libdir: ${base_libdir} - strip .so files in this directory + :param max_process: number of stripping processes started in parallel :param qa_already_stripped: Set to True if already-stripped' in ${INSANE_SKIP} This is for proper logging and messages only. """ @@ -164,7 +165,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp # ...but is it ELF, and is it already stripped? checkelf.append(file) inodecache[file] = s.st_ino - results = oe.utils.multiprocess_launch(is_elf, checkelf, d) + results = oe.utils.multiprocess_launch_mp(is_elf, checkelf, max_process) for (file, elf_file) in results: #elf_file = is_elf(file) if elf_file & 1: @@ -192,7 +193,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp elf_file = int(elffiles[file]) sfiles.append((file, elf_file, strip_cmd)) - oe.utils.multiprocess_launch(runstrip, sfiles, d) + oe.utils.multiprocess_launch_mp(runstrip, sfiles, max_process) def file_translate(file): From patchwork Tue Sep 5 16:51:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B93BCA0FFC for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by mx.groups.io with SMTP id smtpd.web10.27276.1693933180230669180 for ; Tue, 05 Sep 2023 09:59:40 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=iXnEGhvz; spf=pass (domain: gmail.com, ip: 209.85.221.52, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-31c8a710545so2592031f8f.3 for ; Tue, 05 Sep 2023 09:59:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933178; x=1694537978; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jy8i1/A44FHlgQLN08tx4d2NioWfPZimdixvZe0CbjI=; b=iXnEGhvzypJP5RvCBCHCL0SFC/mcDSafIlNtwMGhNHtCDaurcKNLtgk08aITPHdOqq IgYJz2NV9HsAQxh304G88FNkO8rDWYfCrvgSjX4Z8aWnRG/4hp7y/+ZNM9mjqJgIo2Oi 0JlLTXpBJVNOSDatM/XbVDV0Ax6tON76wzVi11rOfDrIYygaeJpR7W98vTSz/fLLNKi1 uI0veTqiUEmg6PfQ74MYdQdeiJm7ZOaqcEsaPSGbFVRFxJswDSVKHImVQaI6W7YPjKdF keU+cNvQbKhk0/nSb5/piqZBUPexJLVLES5PVMmokR3/JepGju2Go3pPFnstNPTVptbb 5rsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933178; x=1694537978; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jy8i1/A44FHlgQLN08tx4d2NioWfPZimdixvZe0CbjI=; b=CAVFuBbbCZG/12fc2JXiWWg/Zm7l7mp8ZOZOSqtPyTq99A7f7nzzw6ZAKve5aKgsmk ZkfxLWOtLePgfNMY5Dl+8rWJ8YL6gekHxw0azfbdxDeCr3AZT9phy+qhVpp2SUD2w4uj UnPya2eQuvlhtYPSZu6pPeEVjB8tf9/pxxZbOk+oUqAVSMmXgiPLqFDcWy2vBeJ/LNCn lzClsoCfV3+KDgjXmTXEcfaYzRy2RY/0DBBv7rzJayYWX2jUUGQnCgU7epcmHY/7PKwr RbiydSxZ64hIyL6xldvb07nt3PvBz1+r6AvGd8NEr5epaP630RG1Gdk2glvf2CRk9tXn jHNw== X-Gm-Message-State: AOJu0YxIjHCrZ61TuVaJzoNwrtfhfjfMrlDbKbpMapYao6wznLxQ0uYI DahnChHHFfMhGd6wH6GK8IAUk8ChLCI= X-Google-Smtp-Source: AGHT+IGnRaMq4C8iX2Zo+1vmuAVYu3Gfc+yGB+1EV/JRGQas1vOUPHyLkEuDJzjn3zDfoK+q1iEDRw== X-Received: by 2002:adf:e407:0:b0:317:3e58:64cb with SMTP id g7-20020adfe407000000b003173e5864cbmr265686wrm.46.1693933178379; Tue, 05 Sep 2023 09:59:38 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:37 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 10/14] devtool: refactor deploy-target Date: Tue, 5 Sep 2023 18:51:19 +0200 Message-ID: <20230905165913.2560907-11-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187242 Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/__init__.py | 5 +- scripts/lib/devtool/deploy.py | 230 +++++++++++++++++--------------- 2 files changed, 124 insertions(+), 111 deletions(-) diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py index 702db669de3..7d64547616e 100644 --- a/scripts/lib/devtool/__init__.py +++ b/scripts/lib/devtool/__init__.py @@ -78,12 +78,15 @@ def exec_fakeroot(d, cmd, **kwargs): """Run a command under fakeroot (pseudo, in fact) so that it picks up the appropriate file permissions""" # Grab the command and check it actually exists fakerootcmd = d.getVar('FAKEROOTCMD') + fakerootenv = d.getVar('FAKEROOTENV') + exec_fakeroot_no_d(fakerootcmd, fakerootenv, cmd, kwargs) + +def exec_fakeroot_no_d(fakerootcmd, fakerootenv, cmd, **kwargs): if not os.path.exists(fakerootcmd): logger.error('pseudo executable %s could not be found - have you run a build yet? pseudo-native should install this and if you have run any build then that should have been built') return 2 # Set up the appropriate environment newenv = dict(os.environ) - fakerootenv = d.getVar('FAKEROOTENV') for varvalue in fakerootenv.split(): if '=' in varvalue: splitval = varvalue.split('=', 1) diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index e14a5874177..ea7e2cb1ae8 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -16,7 +16,7 @@ import bb.utils import argparse_oe import oe.types -from devtool import exec_fakeroot, setup_tinfoil, check_workspace_recipe, DevtoolError +from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, DevtoolError logger = logging.getLogger('devtool') @@ -133,16 +133,11 @@ def _prepare_remote_script(deploy, verbose=False, dryrun=False, undeployall=Fals return '\n'.join(lines) - - -def deploy(args, config, basepath, workspace): - """Entry point for the devtool 'deploy' subcommand""" +def deploy_cached(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args): import math import oe.recipeutils import oe.package - check_workspace_recipe(workspace, args.recipename, checksrc=False) - try: host, destdir = args.target.split(':') except ValueError: @@ -152,116 +147,131 @@ def deploy(args, config, basepath, workspace): if not destdir.endswith('/'): destdir += '/' + recipe_outdir = srcdir + if not os.path.exists(recipe_outdir) or not os.listdir(recipe_outdir): + raise DevtoolError('No files to deploy - have you built the %s ' + 'recipe? If so, the install step has not installed ' + 'any files.' % args.recipename) + + if args.strip and not args.dry_run: + # Fakeroot copy to new destination + srcdir = recipe_outdir + recipe_outdir = os.path.join(workdir, 'devtool-deploy-target-stripped') + if os.path.isdir(recipe_outdir): + exec_fakeroot_no_d(fakerootcmd, fakerootenv, "rm -rf %s" % recipe_outdir, shell=True) + exec_fakeroot_no_d(fakerootcmd, fakerootenv, "cp -af %s %s" % (os.path.join(srcdir, '.'), recipe_outdir), shell=True) + os.environ['PATH'] = ':'.join([os.environ['PATH'], path or '']) + oe.package.strip_execs(args.recipename, recipe_outdir, strip_cmd, libdir, base_libdir, max_process) + + filelist = [] + inodes = set({}) + ftotalsize = 0 + for root, _, files in os.walk(recipe_outdir): + for fn in files: + fstat = os.lstat(os.path.join(root, fn)) + # Get the size in kiB (since we'll be comparing it to the output of du -k) + # MUST use lstat() here not stat() or getfilesize() since we don't want to + # dereference symlinks + if fstat.st_ino in inodes: + fsize = 0 + else: + fsize = int(math.ceil(float(fstat.st_size)/1024)) + inodes.add(fstat.st_ino) + ftotalsize += fsize + # The path as it would appear on the target + fpath = os.path.join(destdir, os.path.relpath(root, recipe_outdir), fn) + filelist.append((fpath, fsize)) + + if args.dry_run: + print('Files to be deployed for %s on target %s:' % (args.recipename, args.target)) + for item, _ in filelist: + print(' %s' % item) + return 0 + + extraoptions = '' + if args.no_host_check: + extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' + if not args.show_status: + extraoptions += ' -q' + + scp_sshexec = '' + ssh_sshexec = 'ssh' + if args.ssh_exec: + scp_sshexec = "-S %s" % args.ssh_exec + ssh_sshexec = args.ssh_exec + scp_port = '' + ssh_port = '' + if args.port: + scp_port = "-P %s" % args.port + ssh_port = "-p %s" % args.port + + if args.key: + extraoptions += ' -i %s' % args.key + + # In order to delete previously deployed files and have the manifest file on + # the target, we write out a shell script and then copy it to the target + # so we can then run it (piping tar output to it). + # (We cannot use scp here, because it doesn't preserve symlinks.) + tmpdir = tempfile.mkdtemp(prefix='devtool') + try: + tmpscript = '/tmp/devtool_deploy.sh' + tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list') + shellscript = _prepare_remote_script(deploy=True, + verbose=args.show_status, + nopreserve=args.no_preserve, + nocheckspace=args.no_check_space) + # Write out the script to a file + with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f: + f.write(shellscript) + # Write out the file list + with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f: + f.write('%d\n' % ftotalsize) + for fpath, fsize in filelist: + f.write('%s %d\n' % (fpath, fsize)) + # Copy them to the target + ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True) + if ret != 0: + raise DevtoolError('Failed to copy script to %s - rerun with -s to ' + 'get a complete error message' % args.target) + finally: + shutil.rmtree(tmpdir) + + # Now run the script + ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, 'tar cf - . | %s %s %s %s \'sh %s %s %s %s\'' % (ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) + if ret != 0: + raise DevtoolError('Deploy failed - rerun with -s to get a complete ' + 'error message') + + logger.info('Successfully deployed %s' % recipe_outdir) + + files_list = [] + for root, _, files in os.walk(recipe_outdir): + for filename in files: + filename = os.path.relpath(os.path.join(root, filename), recipe_outdir) + files_list.append(os.path.join(destdir, filename)) + +def deploy(args, config, basepath, workspace): + """Entry point for the devtool 'deploy' subcommand""" + check_workspace_recipe(workspace, args.recipename, checksrc=False) + tinfoil = setup_tinfoil(basepath=basepath) try: try: rd = tinfoil.parse_recipe(args.recipename) + + srcdir = rd.getVar('D') + workdir = rd.getVar('WORKDIR') + path = rd.getVar('PATH') + strip_cmd = rd.getVar('STRIP') + libdir = rd.getVar('libdir'), + base_libdir = rd.getVar('base_libdir') + max_process = int(rd.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) + fakerootcmd = rd.getVar('FAKEROOTCMD') + fakerootenv = rd.getVar('FAKEROOTENV') + deploy_cached(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args) except Exception as e: raise DevtoolError('Exception parsing recipe %s: %s' % (args.recipename, e)) - recipe_outdir = rd.getVar('D') - if not os.path.exists(recipe_outdir) or not os.listdir(recipe_outdir): - raise DevtoolError('No files to deploy - have you built the %s ' - 'recipe? If so, the install step has not installed ' - 'any files.' % args.recipename) - - if args.strip and not args.dry_run: - # Fakeroot copy to new destination - srcdir = recipe_outdir - recipe_outdir = os.path.join(rd.getVar('WORKDIR'), 'devtool-deploy-target-stripped') - if os.path.isdir(recipe_outdir): - exec_fakeroot(rd, "rm -rf %s" % recipe_outdir, shell=True) - exec_fakeroot(rd, "cp -af %s %s" % (os.path.join(srcdir, '.'), recipe_outdir), shell=True) - os.environ['PATH'] = ':'.join([os.environ['PATH'], rd.getVar('PATH') or '']) - oe.package.strip_execs(args.recipename, recipe_outdir, rd.getVar('STRIP'), rd.getVar('libdir'), - rd.getVar('base_libdir'), rd) - - filelist = [] - inodes = set({}) - ftotalsize = 0 - for root, _, files in os.walk(recipe_outdir): - for fn in files: - fstat = os.lstat(os.path.join(root, fn)) - # Get the size in kiB (since we'll be comparing it to the output of du -k) - # MUST use lstat() here not stat() or getfilesize() since we don't want to - # dereference symlinks - if fstat.st_ino in inodes: - fsize = 0 - else: - fsize = int(math.ceil(float(fstat.st_size)/1024)) - inodes.add(fstat.st_ino) - ftotalsize += fsize - # The path as it would appear on the target - fpath = os.path.join(destdir, os.path.relpath(root, recipe_outdir), fn) - filelist.append((fpath, fsize)) - - if args.dry_run: - print('Files to be deployed for %s on target %s:' % (args.recipename, args.target)) - for item, _ in filelist: - print(' %s' % item) - return 0 - - extraoptions = '' - if args.no_host_check: - extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' - if not args.show_status: - extraoptions += ' -q' - - scp_sshexec = '' - ssh_sshexec = 'ssh' - if args.ssh_exec: - scp_sshexec = "-S %s" % args.ssh_exec - ssh_sshexec = args.ssh_exec - scp_port = '' - ssh_port = '' - if args.port: - scp_port = "-P %s" % args.port - ssh_port = "-p %s" % args.port - - if args.key: - extraoptions += ' -i %s' % args.key - - # In order to delete previously deployed files and have the manifest file on - # the target, we write out a shell script and then copy it to the target - # so we can then run it (piping tar output to it). - # (We cannot use scp here, because it doesn't preserve symlinks.) - tmpdir = tempfile.mkdtemp(prefix='devtool') - try: - tmpscript = '/tmp/devtool_deploy.sh' - tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list') - shellscript = _prepare_remote_script(deploy=True, - verbose=args.show_status, - nopreserve=args.no_preserve, - nocheckspace=args.no_check_space) - # Write out the script to a file - with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f: - f.write(shellscript) - # Write out the file list - with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f: - f.write('%d\n' % ftotalsize) - for fpath, fsize in filelist: - f.write('%s %d\n' % (fpath, fsize)) - # Copy them to the target - ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True) - if ret != 0: - raise DevtoolError('Failed to copy script to %s - rerun with -s to ' - 'get a complete error message' % args.target) - finally: - shutil.rmtree(tmpdir) - - # Now run the script - ret = exec_fakeroot(rd, 'tar cf - . | %s %s %s %s \'sh %s %s %s %s\'' % (ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) - if ret != 0: - raise DevtoolError('Deploy failed - rerun with -s to get a complete ' - 'error message') - - logger.info('Successfully deployed %s' % recipe_outdir) - - files_list = [] - for root, _, files in os.walk(recipe_outdir): - for filename in files: - filename = os.path.relpath(os.path.join(root, filename), recipe_outdir) - files_list.append(os.path.join(destdir, filename)) finally: tinfoil.shutdown() From patchwork Tue Sep 5 16:51:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 098DBCA0FFA for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web10.27278.1693933181106658936 for ; Tue, 05 Sep 2023 09:59:41 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=Zubd6Gog; spf=pass (domain: gmail.com, ip: 209.85.221.47, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f47.google.com with SMTP id ffacd0b85a97d-31ad9155414so2141338f8f.3 for ; Tue, 05 Sep 2023 09:59:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933179; x=1694537979; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P1SnTuAhFxDpsGGX8wM7sKL+0XZkEv7Dw30w3L3RU5E=; b=Zubd6GogDZKyM3TIc8Ax3QpKVMF0Z+bsvT5L8f7PJSe1tIWTTgOGsVmiScCpuxa6cl 61WYsTaLSE/VhXDIKUtQpls51RSlAQc6GNyrkU6FDzzlSyTP0bJ4wdlSpwAEV0Cdw0LW HeLk2feVlXjIUw5D8zbJ/y7lgDYPxe7cyl1mW7PJEd3+/+PGrpcN3u8INvdBvZcBZE8f kow4q744do9DHOMX4nRfSgoe9vTmuAnJKDczNlmkNWEVKSmOyo6Ua0znwvSMfOtVrI9e aa/0ttnzKZ0Mz+fgSWDTPZuCCAZaE9RH3QsxXlT4izaCkKWEFa132J6RE1ZLn3Tv1bwE Cjkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933179; x=1694537979; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P1SnTuAhFxDpsGGX8wM7sKL+0XZkEv7Dw30w3L3RU5E=; b=Rtsog967mrg1hsqcMGepOWqXX2T46gLH4RC1W68za/yrUlRrJeVBec8tvhEIdVrM1F 8gIPqcp92mtfM4gYd7u/wg3qmiiCf0sOO2WNbzKdWu0h1Gpahzb+DxlN6gFOso7gqo8w wpNzXSLTYeoHonAeS9k6+zFR/FRzbql24lBMafM27phlcjQaG9bbw6SqcPwbhy2thD2B JlZmblSth1eOTwKxhJrVhBn+FJU32Ne6kXq8szdDzVAe0unzybpIKAg8xfuX2ausoAEp M7PlCgYEAkAmQEjMSL9W9/klmqIGK2im87Y6Itss3JnQOG5Ry0OBTlaLmX6KQbI3Rwzr g9tA== X-Gm-Message-State: AOJu0YzHecPH3esIwIs8dHVT8nciULJChdx1syWn/ylCmMfuPzLFviwC AKXBVMlCuhLJy8cLlFBX+c4lCZAXd04= X-Google-Smtp-Source: AGHT+IHwDmKonaYT9Hf+vqCZNTYLLe2LhFBlzl36HYMxl853EzoDCMdvy6Rq4AJ2bzZ+5NakXmHT5Q== X-Received: by 2002:a5d:5450:0:b0:31c:84e9:20fd with SMTP id w16-20020a5d5450000000b0031c84e920fdmr367454wrv.13.1693933179319; Tue, 05 Sep 2023 09:59:39 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:38 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 11/14] devtool: ide make deploy-target quicker Date: Tue, 5 Sep 2023 18:51:20 +0200 Message-ID: <20230905165913.2560907-12-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187243 Instead of calling devtool deploy-target which starts a bitbake server to get some variables the previous refactoring allows to generate a simple script which does no longer depend on variables from bitbake. This is much faster. --- scripts/lib/devtool/ide.py | 115 ++++++++++++++++++------------------- 1 file changed, 55 insertions(+), 60 deletions(-) diff --git a/scripts/lib/devtool/ide.py b/scripts/lib/devtool/ide.py index 7c7040232c4..9c724509d48 100755 --- a/scripts/lib/devtool/ide.py +++ b/scripts/lib/devtool/ide.py @@ -9,6 +9,7 @@ import os import stat +import sys import logging import json import re @@ -360,15 +361,19 @@ class RecipeModified: self.bbappend = None # recipe variables from d.getVar self.b = None + self.base_libdir = None self.bpn = None self.d = None self.fakerootcmd = None self.fakerootenv = None + self.libdir = None + self.max_process = None self.package_arch = None self.path = None self.recipe_sysroot = None self.recipe_sysroot_native = None self.staging_incdir = None + self.strip_cmd = None self.target_arch = None self.workdir = None # replicate bitbake build environment @@ -387,11 +392,6 @@ class RecipeModified: self.meson_cross_file = None # vscode self.dot_code_dir = None - # TODO: remove calling devtool - self.bb_env_passthrough_additions = None - self.bbpath = None - self.bitbakepath = None - self.topdir = None def initialize(self, config, workspace, tinfoil): recipe_d = parse_recipe( @@ -413,10 +413,14 @@ class RecipeModified: shutil.rmtree(self.temp_dir) self.b = recipe_d.getVar('B') + self.base_libdir = recipe_d.getVar('base_libdir') self.bpn = recipe_d.getVar('BPN') self.d = recipe_d.getVar('D') self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD') self.fakerootenv = recipe_d.getVar('FAKEROOTENV') + self.libdir = recipe_d.getVar('libdir'), + self.max_process = int(recipe_d.getVar( + "BB_NUMBER_THREADS") or os.cpu_count() or 1) self.package_arch = recipe_d.getVar('PACKAGE_ARCH') self.path = recipe_d.getVar('PATH') self.recipe_sysroot = os.path.realpath( @@ -425,15 +429,10 @@ class RecipeModified: recipe_d.getVar('RECIPE_SYSROOT_NATIVE')) self.staging_incdir = os.path.realpath( recipe_d.getVar('STAGING_INCDIR')) + self.strip_cmd = recipe_d.getVar('STRIP') self.target_arch = recipe_d.getVar('TARGET_ARCH') self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR')) - self.bb_env_passthrough_additions = recipe_d.getVar( - 'BB_ENV_PASSTHROUGH_ADDITIONS') - self.bbpath = recipe_d.getVar('BBPATH') - self.bitbakepath = recipe_d.getVar('BITBAKEPATH') - self.topdir = recipe_d.getVar('TOPDIR') - self.__init_exported_variables(recipe_d) if bb.data.inherits_class('cmake', recipe_d): @@ -971,6 +970,38 @@ class RecipeModified: cmd_lines.append(install_cmd) return self.write_script(cmd_lines, 'bb_run_do_install') + def gen_deploy_target_script(self, args): + """Generate a quicker (works without tinfoil) variant of devtool target-deploy""" + cmd_lines = ['#!/usr/bin/env python3'] + cmd_lines.append('import sys') + cmd_lines.append('devtool_sys_path = %s' % str(sys.path)) + cmd_lines.append('devtool_sys_path.reverse()') + cmd_lines.append('for p in devtool_sys_path:') + cmd_lines.append(' if p not in sys.path:') + cmd_lines.append(' sys.path.insert(0, p)') + cmd_lines.append('from devtool.deploy import deploy_cached') + args_filter = ['debug', 'dry_run', 'key', 'no_check_space', 'no_host_check', + 'no_preserve', 'port', 'recipename', 'show_status', 'ssh_exec', 'strip', 'target'] + filtered_args_dict = {key: value for key, value in vars( + args).items() if key in args_filter} + cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict)) + cmd_lines.append('class Dict2Class(object):') + cmd_lines.append(' def __init__(self, my_dict):') + cmd_lines.append(' for key in my_dict:') + cmd_lines.append(' setattr(self, key, my_dict[key])') + cmd_lines.append('filtered_args = Dict2Class(filtered_args_dict)') + cmd_lines.append('deploy_cached("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args)' % + (self.d, self.workdir, self.path, self.strip_cmd, + self.libdir, self.base_libdir, self.max_process, + self.fakerootcmd, self.fakerootenv)) + return self.write_script(cmd_lines, 'deploy_target') + + def gen_install_deploy_script(self, args): + cmd_lines = ['#!/bin/sh'] + cmd_lines.append(self.gen_fakeroot_install_script()) + cmd_lines.append(self.gen_deploy_target_script(args)) + return self.write_script(cmd_lines, 'install_and_deploy') + def write_script(self, cmd_lines, script_name): bb.utils.mkdirhier(self.temp_dir) script_file = os.path.join(self.temp_dir, script_name) @@ -981,61 +1012,15 @@ class RecipeModified: return script_file def vscode_tasks(self, args, gdb_cross): - run_do_install = self.gen_fakeroot_install_script() - pythonpath_new = os.path.realpath( - os.path.join(self.bitbakepath, '../lib')) - try: - pythonpath = os.environ['PYTONPATH'].split(':') - if pythonpath_new not in pythonpath: - pythonpath = pythonpath_new + ":" + pythonpath - except KeyError: - pythonpath = pythonpath_new - install_label = "do_install %s" % self.bpn - deploy_label = "devtool deploy-target %s" % self.bpn - bb_env = { - # "BBPATH": self.bbpath, - "PYTHONPATH": pythonpath, - "BUILDDIR": self.topdir, - # Use e.g. Python from host, not python-native - "PATH": os.environ['PATH'], - "BB_ENV_PASSTHROUGH_ADDITIONS": self.bb_env_passthrough_additions - } - bb_options = { - "cwd": self.topdir - } + run_install_deploy = self.gen_install_deploy_script(args) tasks_dict = { "version": "2.0.0", "tasks": [ { - "label": install_label, - "type": "shell", - "command": run_do_install, - "problemMatcher": [] - }, - { - "label": deploy_label, + "label": "install && deploy-target %s" % self.bpn, "type": "shell", - "command": "devtool", # TODO: Generate a self contained script which does what devtool target-deploy does but without connection to the bitbake server - "args": ["--bbpath", self.bbpath, "deploy-target", self.bpn, args.target, "--strip", "--no-host-check"], - "linux": { - "options": { - "env": bb_env - }}, - "options": bb_options, + "command": run_install_deploy, "problemMatcher": [] - }, - { - "label": "install && deploy-target %s" % self.bpn, - "dependsOrder": "sequence", - "dependsOn": [ - install_label, - deploy_label - ], - "problemMatcher": [], - "group": { - "kind": "build", - "isDefault": True - } } ] } @@ -1183,6 +1168,16 @@ def register_commands(subparsers, context): '--skip-bitbake', help='Generate IDE configuration but skip calling bibtake to update the SDK.', action='store_true') parser_ide.add_argument( '-k', '--bitbake-k', help='Pass -k parameter to bitbake', action='store_true') + parser_ide.add_argument( + '--no-strip', help='Do not strip executables prior to deploy', dest='strip', action='store_false') + parser_ide.add_argument( + '-n', '--dry-run', help='List files to be undeployed only', action='store_true') + parser_ide.add_argument( + '-s', '--show-status', help='Show progress/status output', action='store_true') + parser_ide.add_argument( + '-p', '--no-preserve', help='Do not preserve existing files', action='store_true') + parser_ide.add_argument( + '--no-check-space', help='Do not check for available space before deploying', action='store_true') parser_ide.add_argument( '--debug-build-config', help='Use debug build flags, for example set CMAKE_BUILD_TYPE=Debug', action='store_true') parser_ide.set_defaults(func=ide_setup) From patchwork Tue Sep 5 16:51:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAE82CA0FF8 for ; Tue, 5 Sep 2023 16:59:43 +0000 (UTC) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mx.groups.io with SMTP id smtpd.web11.27506.1693933182129853152 for ; Tue, 05 Sep 2023 09:59:42 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=fT8gwW8R; spf=pass (domain: gmail.com, ip: 209.85.128.47, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-402c1407139so28920855e9.1 for ; Tue, 05 Sep 2023 09:59:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933180; x=1694537980; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sv7+Ku4KF5JgOBQ3HF31jwDkk7kvG+2IIEvPIX8kbRU=; b=fT8gwW8RXhly/vf6kiBpD8dtiNg5KnB4GYiFJDLa0Pm9bONIXSNjDAVtYdh+yDRxL1 UvNSy0AZgr/pULP6tQjkJE2AzfbT9ke4Y8qSoPzA6DnzbpcMsniEMRCXO1dbFr0G/4aG w/xJ2g3zxnRGCmz9dcVyV6hZAE/PgX20/HUuT+MtPQLoBYXKcJffObx+ZdDymA/9tq9W k11YBd7SY+EpI7geB8ln1lIq0tb4tzKBxYb+VU/ZVd4m2svRoGesNrK1IwEXHibW82NB x6L87+QPNteBbmMrHqHvkOPFq5WNGH7HDydLYSBxuBkPRyVc2fZDvnAY3jbdyzrGS0AC SeCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933180; x=1694537980; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sv7+Ku4KF5JgOBQ3HF31jwDkk7kvG+2IIEvPIX8kbRU=; b=fmZ/n+NtX+bgi00a997gI3Ku4PMm291n0C8YPry6yo4tBD7/cnF8f4xR0F38I+H52W LrSd/NAPLANAoI8MKA1NwhWDf6wSu//ufLHkQOSa8eoEFmb+4sj2n/pDzljXrdtUvHp8 bV3LioMOO5kazy6qSfb5CNrg4UQVLjbNzqmJmc4WzSURbwL/TxIqhJIVSUhbm95UYSuG k/cus5NRp+vv4YGWw6ejwAixwgZFCd3b3XdKU+/adlyDC3Wclff/In4f/EsBNeqUWzBL +mLjsIVqY/f76qO7iTtSANqe9kEB6E2V5v9tpKUBiqrcB21Ur5ljdIWK4QVL5PQdCNR/ nZGg== X-Gm-Message-State: AOJu0YzRSaPuCRTrHDGoBA+JZU03uufb4cyh649WU8YQPY3isKXmvuyv 8cwruclzOjsGzVR+r2JEd0zna7CWPVA= X-Google-Smtp-Source: AGHT+IGHobYKEBwnAZdA0X065UGIoe5Qa2veTWXV4JL2GywSIdwYcpbKvueGcRk7pZAm9kd3H+46Vg== X-Received: by 2002:a05:600c:210f:b0:3fe:1871:1826 with SMTP id u15-20020a05600c210f00b003fe18711826mr235403wml.27.1693933180262; Tue, 05 Sep 2023 09:59:40 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:39 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 12/14] oe-selftest devtool: refactor runqemu pre-requisites Date: Tue, 5 Sep 2023 18:51:21 +0200 Message-ID: <20230905165913.2560907-13-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187244 Split runqemu pre-requisites into a function which can be re-used by other tests as well. Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 48 +++++++++++++------------ 1 file changed, 26 insertions(+), 22 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index a2b77e528de..b577f6d62a1 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -256,6 +256,31 @@ class DevtoolTestCase(OESelftestTestCase): if remaining_removelines: self.fail('Expected removed lines not found: %s' % remaining_removelines) + def _check_runqemu_prerequisites(self): + """Check runqemu is available + + Whilst some tests would seemingly be better placed as a runtime test, + unfortunately the runtime tests run under bitbake and you can't run + devtool within bitbake (since devtool needs to run bitbake itself). + Additionally we are testing build-time functionality as well, so + really this has to be done as an oe-selftest test. + """ + machine = get_bb_var('MACHINE') + if not machine.startswith('qemu'): + self.skipTest('This test only works with qemu machines') + if not os.path.exists('/etc/runqemu-nosudo'): + self.skipTest('You must set up tap devices with scripts/runqemu-gen-tapdevs before running this test') + result = runCmd('PATH="$PATH:/sbin:/usr/sbin" ip tuntap show', ignore_status=True) + if result.status != 0: + result = runCmd('PATH="$PATH:/sbin:/usr/sbin" ifconfig -a', ignore_status=True) + if result.status != 0: + self.skipTest('Failed to determine if tap devices exist with ifconfig or ip: %s' % result.output) + for line in result.output.splitlines(): + if line.startswith('tap'): + break + else: + self.skipTest('No tap devices found - you must set up tap devices with scripts/runqemu-gen-tapdevs before running this test') + def _test_devtool_add_git_url(self, git_url, version, pn, resulting_src_uri): self.track_for_cleanup(self.workspacedir) self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') @@ -1616,28 +1641,7 @@ class DevtoolExtractTests(DevtoolBase): @OETestTag("runqemu") def test_devtool_deploy_target(self): - # NOTE: Whilst this test would seemingly be better placed as a runtime test, - # unfortunately the runtime tests run under bitbake and you can't run - # devtool within bitbake (since devtool needs to run bitbake itself). - # Additionally we are testing build-time functionality as well, so - # really this has to be done as an oe-selftest test. - # - # Check preconditions - machine = get_bb_var('MACHINE') - if not machine.startswith('qemu'): - self.skipTest('This test only works with qemu machines') - if not os.path.exists('/etc/runqemu-nosudo'): - self.skipTest('You must set up tap devices with scripts/runqemu-gen-tapdevs before running this test') - result = runCmd('PATH="$PATH:/sbin:/usr/sbin" ip tuntap show', ignore_status=True) - if result.status != 0: - result = runCmd('PATH="$PATH:/sbin:/usr/sbin" ifconfig -a', ignore_status=True) - if result.status != 0: - self.skipTest('Failed to determine if tap devices exist with ifconfig or ip: %s' % result.output) - for line in result.output.splitlines(): - if line.startswith('tap'): - break - else: - self.skipTest('No tap devices found - you must set up tap devices with scripts/runqemu-gen-tapdevs before running this test') + self._check_runqemu_prerequisites() self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') # Definitions testrecipe = 'mdadm' From patchwork Tue Sep 5 16:51:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5E85CA0FE2 for ; Tue, 5 Sep 2023 16:59:43 +0000 (UTC) Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by mx.groups.io with SMTP id smtpd.web10.27279.1693933182995975652 for ; Tue, 05 Sep 2023 09:59:43 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=TAnhdykQ; spf=pass (domain: gmail.com, ip: 209.85.128.54, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-401d2e11dacso337485e9.0 for ; Tue, 05 Sep 2023 09:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933181; x=1694537981; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AQGicNVlvlsWmQlA3v7P1PEqE3sAln7UuN/ygkc/1yo=; b=TAnhdykQhSGVFOafX6bZu5mZwyqT2afIfeIbcFUHLBnHMGTQmLFlW5SNsQPONFReA2 q8L74aA702KUPVhL8uqiEIPmY377QSCAEAJuwO2094WTtYwYISxcGYqB+VjT85bDHjaD BxCdIHD0gC68sxz7ww3SKfu49rigysb7DJ9Kz3acObN7P4VgslsbIPbsNL3ATTNpqJRQ ldGak0H1nYbcVz2eRo6qTens/JSrF/TbexrJMyZK71gMzQD9EAk80choTaotmhiW9sD7 VqkrqIqbPHo7X93Gh+3rkMZypFU+whkNA4JLV0mKVYfUn92k5Zq15sdkS0VFrYTQ6v+O n3oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933181; x=1694537981; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AQGicNVlvlsWmQlA3v7P1PEqE3sAln7UuN/ygkc/1yo=; b=GOEfrHz0M20MPwcxQQKRRjI0Pzz8+H6sv6vbKCr9VH680xdBnoobYgdorOnPf2iRjF 0F/Y+xs3Twa7KzbsNbe6BZDJROoWyuUuemz06xQomUGuGbyDyIAOy4bci1ZvrV7BRFcc T++Ww2+tIPDGBPuBB80uwah33FJNqJyfZDG8TCCG7wPV6LPsyXsfApdSrI6HVRq7ajlo T1TnpWWJ6d2qoRL+A1M/AhnI3WMxqBE+UnesL40Za8Y8nL8upFLvLnxTeA15sWlufHGx IKlprXEGsodNmJeYNY0uN8bYOwu4giD2UwsM5bTbcwMad4Zmoa3qnmtVKC6hjVKoo2Od EwXQ== X-Gm-Message-State: AOJu0YxVTZExshuXiFpBsLpmGGzH81S/0n5jMJ1tYThDD5xVuNqXLBFc t3nz/u6ZMhnB4dk3BNB6wumQtcuumBc= X-Google-Smtp-Source: AGHT+IH8pZeB2odHV2GuqOWygq/+GhzBAm8l9o5MerJdzAYrnxrR8xPl1mdnN2g2AmU2h5aYVHTdDA== X-Received: by 2002:a5d:49c6:0:b0:317:dcdd:3fa1 with SMTP id t6-20020a5d49c6000000b00317dcdd3fa1mr358695wrs.34.1693933181077; Tue, 05 Sep 2023 09:59:41 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:40 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 13/14] oe-selftest devtool: ide tests Date: Tue, 5 Sep 2023 18:51:22 +0200 Message-ID: <20230905165913.2560907-14-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:43 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187245 Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 179 ++++++++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index b577f6d62a1..3bfc9510109 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -11,6 +11,7 @@ import tempfile import glob import fnmatch import unittest +import json from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer @@ -2196,3 +2197,181 @@ class DevtoolUpgradeTests(DevtoolBase): #Step 4.5 runCmd("grep %s %s" % (modconfopt, codeconfigfile)) + + + +class DevtoolIdeTests(DevtoolBase): + def __devtool_ide_recipe(self, recipe_name, build_file, testimage): + self._check_runqemu_prerequisites() + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake -c clean %s' % recipe_name) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + + conf_lines = [ + 'IMAGE_CLASSES += "image-combined-dbg"', + 'IMAGE_GEN_DEBUGFS = "1"', + 'IMAGE_INSTALL += "gdbserver %s-ptest"' % recipe_name + ] + self.write_config("\n".join(conf_lines)) + + result = runCmd('devtool modify %s -x %s' % (recipe_name, tempdir)) + self.assertExists(os.path.join(tempdir, build_file), + 'Extracted source could not be found') + self.assertExists(os.path.join(self.workspacedir, 'conf', + 'layer.conf'), 'Workspace directory not created') + matches = glob.glob(os.path.join(self.workspacedir, + 'appends', recipe_name + '.bbappend')) + self.assertTrue(matches, 'bbappend not created %s' % result.output) + + # Test devtool status + result = runCmd('devtool status') + self.assertIn(recipe_name, result.output) + self.assertIn(tempdir, result.output) + self._check_src_repo(tempdir) + + # Usually devtool ide would initiate the build of the SDK. + # But there is a circular dependency with starting Qemu and passing the IP of runqemu to devtool ide. + bitbake("%s qemu-native qemu-helper-native" % testimage) + deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE') + self.add_command_to_tearDown('bitbake -c clean %s' % testimage) + self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, testimage)) + + return tempdir + + def __devtool_ide_qemu(self, tempdir, qemu, recipe_name, bitbake_sdk_cmd, example_exe): + # Verify do_install works + with open(os.path.join(tempdir, '.vscode', 'tasks.json')) as tasks_j: + tasks_d = json.load(tasks_j) + tasks = tasks_d["tasks"] + task_install = next((task for task in tasks if task["label"] == "install && deploy-target %s" % recipe_name), None) + self.assertIsNot(task_install, None) + install_deploy_cmd = task_install["command"] + # execute only the bb_run_do_install script since the deploy would require e.g. Qemu running. + install_cmd = install_deploy_cmd.replace("install_and_deploy", "bb_run_do_install") + runCmd(install_cmd, cwd=tempdir) + + # Verify if re-building the SDK still works and the deploy and install script gets generated + runCmd(bitbake_sdk_cmd) + self.assertExists(install_deploy_cmd, 'install_and_deploy script not found') + + MAGIC_STRING_ORIG = "Magic: 123456789" + MAGIC_STRING_NEW = "Magic: 987654321" + ptest_cmd = "ptest-runner " + recipe_name + + # Verify the unmodified example prints the magic string + status, output = qemu.run(example_exe) + self.assertEqual(status, 0, msg="%s failed: %s" % (example_exe, output)) + self.assertIn(MAGIC_STRING_ORIG, output) + + # Verify the unmodified ptests work + status, output = qemu.run(ptest_cmd) + self.assertEqual(status, 0, msg="%s failed: %s" % (ptest_cmd, output)) + self.assertIn("PASS: cpp-example-lib", output) + + # Replace the Magic String in the code, compile and deploy to Qemu + cpp_example_lib_hpp = os.path.join(tempdir, 'cpp-example-lib.hpp') + with open(cpp_example_lib_hpp, 'r') as file: + cpp_code = file.read() + cpp_code = cpp_code.replace(MAGIC_STRING_ORIG, MAGIC_STRING_NEW) + with open(cpp_example_lib_hpp, 'w') as file: + file.write(cpp_code) + runCmd(install_deploy_cmd, cwd=tempdir) + + # Verify the modified example prints the modified magic string + status, output = qemu.run(example_exe) + self.assertEqual(status, 0, msg="%s failed: %s" % (example_exe, output)) + self.assertNotIn(MAGIC_STRING_ORIG, output) + self.assertIn(MAGIC_STRING_NEW, output) + + # Verify the modified example ptests work + status, output = qemu.run(ptest_cmd) + self.assertEqual(status, 0, msg="%s failed: %s" % (ptest_cmd, output)) + self.assertIn("PASS: cpp-example-lib", output) + + @OETestTag("runqemu") + def test_devtool_ide_recipe_cmake(self): + recipe_name = "cmake-example" + example_exe = "cmake-example" + build_file = "CMakeLists.txt" + testimage = "oe-selftest-image" + + tempdir = self.__devtool_ide_recipe(recipe_name, build_file, testimage) + + # Verify deployment to Qemu works + with runqemu(testimage) as qemu: + bitbake_sdk_cmd = 'devtool ide %s %s -t root@%s -c --ide=code' % (recipe_name, testimage, qemu.ip) + runCmd(bitbake_sdk_cmd) + + with open(os.path.join(tempdir, 'CMakeUserPresets.json')) as cmake_preset_j: + cmake_preset_d = json.load(cmake_preset_j) + config_presets = cmake_preset_d["configurePresets"] + self.assertEqual(len(config_presets), 1) + cmake_exe = config_presets[0]["cmakeExecutable"] + preset_name = config_presets[0]["name"] + + # Verify the wrapper for cmake native is available + self.assertExists(cmake_exe) + + # Verify the cmake preset generated by devtool ide is available + result = runCmd('%s --list-presets' % cmake_exe, cwd=tempdir) + self.assertIn(preset_name, result.output) + + # Verify cmake re-uses the o files compiled by bitbake + result = runCmd('%s --build --preset %s' % (cmake_exe, preset_name), cwd=tempdir) + self.assertIn("ninja: no work to do.", result.output) + + # Verify the unit tests work (in Qemu) + result = runCmd('%s --build --preset %s --target test' % (cmake_exe, preset_name), cwd=tempdir) + self.assertIn("100% tests passed", result.output) + + # Verify re-building and testing works again + result = runCmd('%s --build --preset %s --target clean' % (cmake_exe, preset_name), cwd=tempdir) + self.assertIn("Cleaning all built files...", result.output) + result = runCmd('%s --build --preset %s' % (cmake_exe, preset_name), cwd=tempdir) + self.assertIn("Building", result.output) + self.assertIn("Linking", result.output) + result = runCmd('%s --build --preset %s --target test' % (cmake_exe, preset_name), cwd=tempdir) + self.assertIn("Running tests...", result.output) + self.assertIn("100% tests passed", result.output) + + self.__devtool_ide_qemu(tempdir, qemu, recipe_name, bitbake_sdk_cmd, example_exe) + + @OETestTag("runqemu") + def test_devtool_ide_recipe_meson(self): + recipe_name = "meson-example" + example_exe = "mesonex" + build_file = "meson.build" + testimage = "oe-selftest-image" + + tempdir = self.__devtool_ide_recipe(recipe_name, build_file, testimage) + + # Verify deployment to Qemu works + with runqemu(testimage) as qemu: + bitbake_sdk_cmd = 'devtool ide %s %s -t root@%s -c --ide=code' % (recipe_name, testimage, qemu.ip) + runCmd(bitbake_sdk_cmd) + + with open(os.path.join(tempdir, '.vscode', 'settings.json')) as settings_j: + settings_d = json.load(settings_j) + meson_exe = settings_d["mesonbuild.mesonPath"] + meson_build_folder = settings_d["mesonbuild.buildFolder"] + + # Verify the wrapper for meson native is available + self.assertExists(meson_exe) + + # Verify meson re-uses the o files compiled by bitbake + result = runCmd('%s compile -C %s' % (meson_exe, meson_build_folder), cwd=tempdir) + self.assertIn("ninja: no work to do.", result.output) + + # Verify the unit tests work (in Qemu) + runCmd('%s test -C %s' % (meson_exe, meson_build_folder), cwd=tempdir) + + # Verify re-building and testing works again + result = runCmd('%s compile -C %s --clean' % (meson_exe, meson_build_folder), cwd=tempdir) + self.assertIn("Cleaning...", result.output) + result = runCmd('%s compile -C %s' % (meson_exe, meson_build_folder), cwd=tempdir) + self.assertIn("Linking target %s" % example_exe, result.output) + runCmd('%s test -C %s' % (meson_exe, meson_build_folder), cwd=tempdir) + + self.__devtool_ide_qemu(tempdir, qemu, recipe_name, bitbake_sdk_cmd, example_exe) From patchwork Tue Sep 5 16:51:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 30004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50632CA1008 for ; Tue, 5 Sep 2023 16:59:44 +0000 (UTC) Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by mx.groups.io with SMTP id smtpd.web11.27508.1693933183446393178 for ; Tue, 05 Sep 2023 09:59:43 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=SZJ99x1T; spf=pass (domain: gmail.com, ip: 209.85.221.43, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-31ad9155414so2141377f8f.3 for ; Tue, 05 Sep 2023 09:59:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693933182; x=1694537982; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AoenDSfxEhrJSXJzwNF0qUZi4ZdzROlnUR735GqJdd4=; b=SZJ99x1Tvv0lfdIEnmiw8OLmmcM87Qmy7H3sv4ruE4sA3vrJi6cNdYdkvp0018+COj 85RLf38Js1nIHJMetMFlGcpRRM2It3IriPe77EcBXk02XqkvT2Ty5MatzR4micE4SxWj WJ6crfhU6KRhOMbAciP81mpWfHaQEK2nl1LpNh6S5b6418iIWV9dYtkjqp+uwZz9W60d sWD6DsvQaHN9fQQpyApxwk4YoOCtRBNGAWSLyPJSFiY+8GpXevyLKiK0tmdI0glSSUT4 Q0cYzgXapKE0XS4YOJGTulWhCxTJI5QxqOE2eFXfQc1em9pfd/m4/T7saVgiYecsL3FG 1eXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693933182; x=1694537982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AoenDSfxEhrJSXJzwNF0qUZi4ZdzROlnUR735GqJdd4=; b=FCm/rK1FPg2ZBjcvA//xkJug5PIu14keblkNdd05sc59jGcoEkTVUC0R64CuSJi+JH 1v4iY4i7hUxHa18p6F08tk4i0eBdzAyvf9NKCHVztqBYS7iZOcasqyr6SjlJ83LBqbLP KzgHorBQ4CGRrXnOTi8O38HdgKhydgCRdGXtk2dGSUlA/GrEsGzuMa0FVmIP7+YDcHzk rDyNDXAiAYACeJIr15LnLSVC6nwGZMa5gqgpQ/edEYCVqkdV7qI19+2Bjuy/93H8kn5L MDK2TO1JtFmWp+JurstRluS4TKfUPpzrwTM5/tTG5xVXTiPU7bPEnWwbQAdz34ZRx3qW ShQg== X-Gm-Message-State: AOJu0YxuhIb63K2c9aeU0c8CpQX2UkbewRyhO0GaZEvBtNMckco3KKk0 1w4VXjJOEHRfpbaORS+0q4PW4S3/aEU= X-Google-Smtp-Source: AGHT+IEK05y7OhDOmZUJhRDe8Auo9Y+Q5BqTfauzv5bnjnteqEl80b30FWMINrF+Iu4v/2kn3tOIDw== X-Received: by 2002:a5d:5450:0:b0:31c:84e9:20fd with SMTP id w16-20020a5d5450000000b0031c84e920fdmr367525wrv.13.1693933181674; Tue, 05 Sep 2023 09:59:41 -0700 (PDT) Received: from t14s-af.fritz.box ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id c7-20020a5d4147000000b0031de43fe9bfsm18146269wrq.0.2023.09.05.09.59.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 09:59:41 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [OE-core][PATCH v5 14/14] docs: cover devtool ide Date: Tue, 5 Sep 2023 18:51:23 +0200 Message-ID: <20230905165913.2560907-15-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230905165913.2560907-1-adrian.freihofer@siemens.com> References: <20230905165913.2560907-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 05 Sep 2023 16:59:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187246 Signed-off-by: Adrian Freihofer --- documentation/sdk-manual/extensible.rst | 97 ++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/documentation/sdk-manual/extensible.rst b/documentation/sdk-manual/extensible.rst index 9e08e57a4e7..d05d4e36aa7 100644 --- a/documentation/sdk-manual/extensible.rst +++ b/documentation/sdk-manual/extensible.rst @@ -237,7 +237,7 @@ all the commands. devtool quick reference. -Three ``devtool`` subcommands provide entry-points into +Four ``devtool`` subcommands provide entry-points into development: - *devtool add*: Assists in adding new software to be built. @@ -245,6 +245,8 @@ development: - *devtool modify*: Sets up an environment to enable you to modify the source of an existing component. +- *devtool ide*: Generates a configuration for an IDE. + - *devtool upgrade*: Updates an existing recipe so that you can build it for an updated set of source files. @@ -632,6 +634,99 @@ command: proceed with your work. If you do use this command, realize that the source tree is preserved. +Use ``devtool ide`` to generate an configuration for the IDE +------------------------------------------------------------ + +``devtool ide`` automatically configures the IDE for cross-compiling and remote debugging. +The IDE is configured to call for example cmake directly. This has several advantages. +First of all it is much faster than using e.g. ``devtool build``. But it also allows to +use the very good integration of build tools like cmake or gdb with VSCode directly. + +Two different use cases are supported: + +- Generate the IDE configuration for a workspace created by ``devtool modify``. + +- Generate the IDE configuration for using a cross-toolchain as provided by + ``bitbake meta-ide-support build-sysroots``. + +Assuming the development environment is set up correctly and a workspace has been created +for the recipe using ``devtool modify recipe``, the following command can create the +configuration for VSCode in the recipe workspace: + + $ devtool ide recipe core-image-minimal --target root@192.168.7.2 + +What this command does exactly depends on the recipe or the build tool used by the recipe. +Currently, only CMake and Meson are supported natively. + +For a recipe which inherits cmake it does: + +- Prepare the SDK by calling bitbake core-image-minimal, gdb-cross, qemu-native... + +- Generate a cmake-preset with configures cmake to use exactly the same environent and + the same cmake-cache configuration as used by ``bitbake recipe``. The cmake-preset referres + to the per-recipe-sysroot of the recipe. + + Currently Configure, Build and Test presets are supported. Test presets execute the test + binaries with Qemu. + +- Generates a helper script to handle the do_install with pseudo + +- Generates some helper scripts to start the gdbserver on the target device + +- Generates the ``.vscode`` folder containing the following files: + + - c_ccp_properties.json: configure the code navigation + + - extensions.json: Recommend the extensions which are used. + + - launch.json: Provide a configuration for remote debugging with gdb-cross and gdbserver. + The debug-symbols are searched in the build-folder, the per-recipe-sysroot and the rootfs-dbg + folder which is provided by the image. + + - settings.json: confgure the indexer to ignore the build folders + + - tasks.json: Provide some helpers for running + + - do_install and ``devtool deploy-target`` + + - start the gdbserver via ssh + +For a recipe which inherits meson a similar configuration is generated. +Because there is nothing like a meson-preset a wrapper script for meson is generated. + +For some special recipes and use cases a per-recipe-sysroot based SDK is not suitable. +Therefore devtool ide also supports setting up the shared sysroots environment and generating +a IDE configurations referring to the shared sysroots. Recipes leading to a shared sysroot +are for example meta-ide-support or shared-sysroots. Also passing none as a recipe name leads +to a shared sysroot SDK. + + $ devtool ide none core-image-minimal + +In case of a shared sysroot SDK the configuration which gets generated for VSCode exposes the +cross-tool-chain as a cmake-kit. If a cmake project is loaded into VSCode the cross-toolchain +can be selected for compiling. + +The default IDE is VSCode. Some hints about using VSCode: + +- To work with cmake press ``Ctrl + Shift + p``, type cmake. + This will show some possible commands like selecting a cmake preset, compiling or running ctest. + A cmake kit might be activated by ``Ctrl + Shift + p``, type cmake quick start, + if not preset file is in the wokspace. + +- To work with meson press ``Ctrl + Shift + p``, type meson. + This will show some possible commands like compiling or executing the unit tests. + +- For the deployment to the target device, just press ``Ctrl + Shift + p``, type task. + Select the install & deploy task. + +- For remote debugging, switch to the debugging view by pressing the play button with the bug on the left side. + This will provide a green play button with a drop-down list where a debug configuration can be selected. + After selecting one of the generated configurations, press the play button. + +Additionally ``--ide=none`` is supported. +With the none IDE some generic configurations files like .gdbinit files and some helper scripts +are generated. + Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software -------------------------------------------------------------------------------------------------------