From patchwork Mon Aug 21 16:07:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91502EE49AB for ; Mon, 21 Aug 2023 16:11:47 +0000 (UTC) Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by mx.groups.io with SMTP id smtpd.web10.1053.1692634305371299353 for ; Mon, 21 Aug 2023 09:11:45 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=sb8JU3La; spf=pass (domain: gmail.com, ip: 209.85.167.48, mailfrom: adrian.freihofer@gmail.com) Received: by mail-lf1-f48.google.com with SMTP id 2adb3069b0e04-4ff9abf18f9so5171704e87.2 for ; Mon, 21 Aug 2023 09:11:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634303; x=1693239103; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NZ08zWm3YJBW8vpIJ55BBYcDMLEYJPQpozpMQfUvfCg=; b=sb8JU3LaLrlU6CHIn6yepuzfIuhtxpojVXnOzIxrUb7nM5zEY6RCl9f8FkxVxG78by 6xgWERdH3Wnhrpw55nTX6KaMhu/tSvJTNWRCf5jbXGko/UYSGG8GFjdf6tsjY5HaXqgk QxIk3AtmTCRWOgnJwWEf/b1LQ20xccNTEhK7ff44EYfwxMYhzuF+nLo5IcYQ5dKeCUyy 1dYNlvJiYTXoTXZU7s+FNdQAaNlpHz8OSA0n5lB1XIQXxSHxUvpPXxEClwAW4t1b8rXb uDBc9rIL2A6B3MLICOdDjJDnLHXp+iXIS+9jitJEYXBZqoMP9cc3o4i8v71OUHQmRQkr mTFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634303; x=1693239103; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NZ08zWm3YJBW8vpIJ55BBYcDMLEYJPQpozpMQfUvfCg=; b=bDSda9HjExmxOwrGmy7rpHlHMwdDY69f5iugW2OFKWHKxYzJ37GhqGBYqpsJUB0d+Y rnMypxqiDRGQ5NOxRMgm8kpMx27hYdMWsRI3r7ffz4y5Hsg9u6F3WCuT2ebkRK9tIO1V oSSi6hB69qcsMgDgZVymYV/cSr5vZyxM39NOXrpYCz57Z8umFsIaHiGF7eFjXCLbGXpM 4sNYPir70DzoPO0k7VHjiHdRW6Y/pj25Cf58Q3xBUb6uxXxkjAffUy5y8XMXv4HEOPN0 AynQuBe1rykG/YKbZJzSVYzRaDhzrAiDvVg/qKibyT1FKveHAkNIiX5D/oJyNTuOBGrx 9LfQ== X-Gm-Message-State: AOJu0YxKIsGBkbZKjZeG6tmmzozrOUd01pnqSd2e2ST054i5X37lvQ/9 17mzwLdqyrq12J8ULvwc5hAaJBNAJ+U= X-Google-Smtp-Source: AGHT+IG3e+AS8b5l9ZGDTVlmaNW0w+Fkm3dJMQ0aq+NQlPZmqtyvSHzMSvenBCml9Uz7Sn6oe2uyDQ== X-Received: by 2002:ac2:5e3b:0:b0:4ff:8863:be01 with SMTP id o27-20020ac25e3b000000b004ff8863be01mr4451441lfg.8.1692634302836; Mon, 21 Aug 2023 09:11:42 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:42 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 01/14] gitignore: vscode bitbake plugin folder Date: Mon, 21 Aug 2023 18:07:04 +0200 Message-ID: <20230821161120.171956-2-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:47 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186464 The vscode bitbake plugin creates its own build folder to run bitbake independently from other bitbake instances. Signed-off-by: Adrian Freihofer --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 8f48d452dab..ca11ee36de2 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,7 @@ /*.patch /.repo/ /build*/ +vscode-bitbake-build/ pyshtables.py pstage/ scripts/oe-git-proxy-socks From patchwork Mon Aug 21 16:07:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29234 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92DEBC3DA66 for ; Mon, 21 Aug 2023 16:11:47 +0000 (UTC) Received: from mail-wr1-f52.google.com (mail-wr1-f52.google.com [209.85.221.52]) by mx.groups.io with SMTP id smtpd.web10.1054.1692634305538203557 for ; Mon, 21 Aug 2023 09:11:45 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=YjFLk3CT; spf=pass (domain: gmail.com, ip: 209.85.221.52, mailfrom: adrian.freihofer@gmail.com) Received: by mail-wr1-f52.google.com with SMTP id ffacd0b85a97d-31781e15a0cso3197204f8f.3 for ; Mon, 21 Aug 2023 09:11:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634304; x=1693239104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ztC4glNDsCLeZ5ufLXMbEaWJuPeagRJ9R8zvlcJsRUo=; b=YjFLk3CT6sMxBOYGIffTC97XQ1krjwj9NQfqDvFKrBxJcaF1yyVQctTA0yDtAOikqH OIy4aPMKi62IoKhbW/z/KaDdanIfQkn94vPtXIAwMwqTd9kuSiZTokfOpOMTILsQwTwz XcDnn2SjolpYMi3jBE9yYHmv2dTbK/IRMCvOB/7PguJiWGIujxYgJ1AbqEPv8sCIlvvu dfOgtHzBpj3lq7dm/EGuGE5AG+u+rxxP9WUQG5KNoGrbq+Jss+RBxR3jrq0dqJs4xHqq Wi61/f5wYrduLqGLDqu1Dum/wlQHT0rEmNikVffuz7RuEkBTofG466tNN1g+FhrR11RC Thiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634304; x=1693239104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ztC4glNDsCLeZ5ufLXMbEaWJuPeagRJ9R8zvlcJsRUo=; b=OZ8nlbA7A9ZSpDe2XUIySh2XZf0N7QpL24YZsYm+jlgnzex4v+eXTv+mANQo7NCh2l IHuGWSNqsGl/LSQ4MrE8FK3Jv5nz+SQQ7iGqbB5FquMTb9rrNTvVYYf5ZBTrO+CCGh4R pd2iNO+mMELgwMYUBwJGfkeyp4F8P56asAKi7Muru6qQC7uI1xnTOIOxkofdhZ4u8f3Y WFF/Ny+sQ9BL82HH5iUfBHdFpSrF2FmdW+RqZxh+qqW3a+3Iw1zd0i55OfM56SRfpn5W 0HYMbWRg6Q/1iDU6F8yLp+nIs/5FJc8RQDYOaTxv6mg86NZav3qeeOpP8gL+fsb8IVbE aVOw== X-Gm-Message-State: AOJu0Yw67nYyiDP1L5EdPWnF2PUhiPM/1XoE5ZwMCpFshYW22MKoOmEP yyCujtEaYVYLSLRp9qfu1gAHWmNEMb8= X-Google-Smtp-Source: AGHT+IE5SZxqPofQhSduROCsYQ8FU6nQy2Mqf3oCTmhuYPRM2te6pmRUkerbjEE1lgCzHQo2cXCXcw== X-Received: by 2002:a5d:560c:0:b0:31c:488e:ff33 with SMTP id l12-20020a5d560c000000b0031c488eff33mr2628705wrv.58.1692634303508; Mon, 21 Aug 2023 09:11:43 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:43 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 02/14] build: vscode configuration Date: Mon, 21 Aug 2023 18:07:05 +0200 Message-ID: <20230821161120.171956-3-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:47 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186465 This is just my setup. It's not supposed to be merged into poky. However, it could be useful to have something similar in poky because if VSCode tries to index the build folder it runs until the OOM killer stops it. Depending on the configuration of the OOM killer it's also very likely that some other randomly selected tasks get killed. --- .vscode/extensions.json | 11 +++++++++++ .vscode/launch.json | 27 +++++++++++++++++++++++++++ .vscode/settings.json | 30 ++++++++++++++++++++++++++++++ oe-init-build-env | 8 ++++++++ 4 files changed, 76 insertions(+) create mode 100644 .vscode/extensions.json create mode 100644 .vscode/launch.json create mode 100644 .vscode/settings.json diff --git a/.vscode/extensions.json b/.vscode/extensions.json new file mode 100644 index 00000000000..09983dbc0dc --- /dev/null +++ b/.vscode/extensions.json @@ -0,0 +1,11 @@ +{ + "recommendations": [ + "EditorConfig.EditorConfig", + "EugenWiens.bitbake", + "maattdd.gitless", + "ms-python.isort", + "ms-python.python", + "ms-python.vscode-pylance", + "timonwong.shellcheck", + ] +} diff --git a/.vscode/launch.json b/.vscode/launch.json new file mode 100644 index 00000000000..145d2ab3e8c --- /dev/null +++ b/.vscode/launch.json @@ -0,0 +1,27 @@ +{ + // Use IntelliSense to learn about possible attributes. + // Hover to view descriptions of existing attributes. + // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 + "version": "0.2.0", + "configurations": [ + { + "name": "Python: bitbake -T-1 --server-only", + "type": "python", + "request": "launch", + "program": "${workspaceFolder}/bitbake/bin/bitbake", + "args": ["-T-1", "--server-only", "-DDD"], + "cwd": "${workspaceFolder}/build", + "envFile": "${workspaceFolder}/oe-init-build-env", + "console": "integratedTerminal", + "justMyCode": true + }, + { + "name": "Python: Current File", + "type": "python", + "request": "launch", + "program": "${file}", + "console": "integratedTerminal", + "justMyCode": true + } + ] +} \ No newline at end of file diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 00000000000..4b61b35e1aa --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,30 @@ +{ + // Configure glob patterns of file paths to exclude from file watching. Patterns must match on absolute paths (i.e. prefix with ** or the full path to match properly). Changing this setting requires a restart. When you experience Code consuming lots of CPU time on startup, you can exclude large folders to reduce the initial load. + "files.watcherExclude": { + "**/.git/**": true, + "**/build*/cache/**": true, + "**/build*/tmp/**": true, + "**/downloads/**": true, + "**/sstate-cache/**": true, + "**/vscode-bitbake-build/**": true, + "**/workspace/sources/**": true + }, + // Configure glob patterns for excluding files and folders. For example, the files explorer decides which files and folders to show or hide based on this setting. + "files.exclude": { + "**/.git/**": true, + "**/build*/cache/**": true, + "**/build*/tmp/**": true, + "**/downloads/**": true, + "**/sstate-cache/**": true, + "**/vscode-bitbake-build/**": true, + "**/workspace/sources/**": true + }, + "python.analysis.exclude": [ + "**/.git/**", + "**/build*/**", + "**/downloads/**", + "**/sstate-cache/**", + "**/vscode-bitbake-build/**", + "**/workspace/sources/**" + ] +} diff --git a/oe-init-build-env b/oe-init-build-env index 38333ab8582..450ae71788c 100755 --- a/oe-init-build-env +++ b/oe-init-build-env @@ -51,3 +51,11 @@ unset OEROOT [ -z "$BUILDDIR" ] || cd "$BUILDDIR" + +# Generate a minimal env file which can be used e.g. to debug bitbake with a Python debugger +cat << EOFbuildenv > $BUILDDIR/.buildenv +BBPATH="$BBPATH" +BUILDDIR="$BUILDDIR" +PATH="$PATH" +BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_PASSTHROUGH_ADDITIONS" +EOFbuildenv From patchwork Mon Aug 21 16:07:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85744EE49A6 for ; Mon, 21 Aug 2023 16:11:47 +0000 (UTC) Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by mx.groups.io with SMTP id smtpd.web11.1046.1692634306149123289 for ; Mon, 21 Aug 2023 09:11:46 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=hqGeuJMJ; spf=pass (domain: gmail.com, ip: 209.85.208.42, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-52889bc61b6so4512671a12.0 for ; Mon, 21 Aug 2023 09:11:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634304; x=1693239104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XmUX6U4agbW3t1fGqkeP7lfXrBnRaDzuSONNIii4H+Q=; b=hqGeuJMJbm9wg/IbLl73JJEqbJAiHdNivikO+5H/QUfGzWO+ntKSBMNtn9WTitehbn RMOOdbIP/7M1mWgz6PX34esPRuDaTaa/FrBfPD9w6J1OVAGIR4PTPpScs416Eydyyohr 6QeCvcEk70tdinnFuIMMOfYaRR7vRYltxlEcraB/chmp0D4BEjTKi73OUZs0TTTmadoH qZF4TmYdjk6MGrpApwi4LsaORjWyBgnKknQz3WWGbOclyHPIXZQ9e1EvjuO3yzQk0NPa OnFdo9AJAo7Stdk4rqkUa3BjH9J0GUCpYjfV4FvOGRMRyIvykBWuiIhocMYVjDsCY6KD jwkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634304; x=1693239104; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XmUX6U4agbW3t1fGqkeP7lfXrBnRaDzuSONNIii4H+Q=; b=jhRxSAhch1Cq+h15ePbHqsV6rH40GFx54cQUKnOAaf+v6V3Ldb3aZFAQk2bmTStr4u 7Aa9JakDt5lJ/Ucpps7h0sMuvWU6BA74gJiGO2JTHZW9xPPo+EhMlxldbi43KEkEIFZU hb3OIi3dDQf8z+MhBQjftd2fLKq6r3LYBGKpUPEyBZUxyw1jfqKrdw5HBsmzAkK7i39E U1gON1e+53Ep8ygWgciv1sxeHnIOrCtZUlhqdlr3Rzi5wWMCv0kkd6JCYC8kJto9hp1l kWddSQLx08o1STKOedIgBinMfMg8+p6CYL+TAUSnjXO/pBM8t8WUfNpHA5f+MxR/pUif LpYQ== X-Gm-Message-State: AOJu0YyrFEwDmIAeYMTSFA3sDdYfDR+uv+ylW6K9ZtFP+N2q5eV3e7Sd 392HXf8TDFExGTGAamJB0O5/J/NVoJ4= X-Google-Smtp-Source: AGHT+IGHRG6T3KuA/6jQpY+KULfnBMx93b2kmWDTFyjYv4UUqUAdlw9cJ1tFN/jR5V2xjArLvXd/Xw== X-Received: by 2002:a05:6402:3447:b0:523:1004:1ca0 with SMTP id l7-20020a056402344700b0052310041ca0mr6391664edc.5.1692634304115; Mon, 21 Aug 2023 09:11:44 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:43 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 03/14] cmake.bbclass: refactor cmake args Date: Mon, 21 Aug 2023 18:07:06 +0200 Message-ID: <20230821161120.171956-4-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:47 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186466 Make the details of the cmake configure arguments available to d.getVar(). This allows to share them with devtool via tinfoil. Signed-off-by: Adrian Freihofer --- meta/classes-recipe/cmake.bbclass | 43 +++++++++++++++++-------------- 1 file changed, 24 insertions(+), 19 deletions(-) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index 0dda104a695..7c40df4cdbf 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -158,6 +158,29 @@ CONFIGURE_FILES = "CMakeLists.txt" do_configure[cleandirs] = "${@d.getVar('B') if d.getVar('S') != d.getVar('B') else ''}" +OECMAKE_ARGS = "\ + -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \ + -DCMAKE_INSTALL_BINDIR:PATH=${@os.path.relpath(d.getVar('bindir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_SBINDIR:PATH=${@os.path.relpath(d.getVar('sbindir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_LIBEXECDIR:PATH=${@os.path.relpath(d.getVar('libexecdir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_SYSCONFDIR:PATH=${sysconfdir} \ + -DCMAKE_INSTALL_SHAREDSTATEDIR:PATH=${@os.path.relpath(d.getVar('sharedstatedir'), d. getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_LOCALSTATEDIR:PATH=${localstatedir} \ + -DCMAKE_INSTALL_LIBDIR:PATH=${@os.path.relpath(d.getVar('libdir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_INCLUDEDIR:PATH=${@os.path.relpath(d.getVar('includedir'), d.getVar('prefix') + '/')} \ + -DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'), d.getVar('prefix') + '/')} \ + -DPYTHON_EXECUTABLE:PATH=${PYTHON} \ + -DPython_EXECUTABLE:PATH=${PYTHON} \ + -DPython3_EXECUTABLE:PATH=${PYTHON} \ + -DLIB_SUFFIX=${@d.getVar('baselib').replace('lib', '')} \ + -DCMAKE_INSTALL_SO_NO_EXE=0 \ + -DCMAKE_TOOLCHAIN_FILE:FILEPATH=${WORKDIR}/toolchain.cmake \ + -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \ + -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON \ + -DFETCHCONTENT_FULLY_DISCONNECTED=ON \ + -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON \ +" + cmake_do_configure() { if [ "${OECMAKE_BUILDPATH}" ]; then bbnote "cmake.bbclass no longer uses OECMAKE_BUILDPATH. The default behaviour is now out-of-tree builds with B=WORKDIR/build." @@ -178,25 +201,7 @@ cmake_do_configure() { ${OECMAKE_GENERATOR_ARGS} \ $oecmake_sitefile \ ${OECMAKE_SOURCEPATH} \ - -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \ - -DCMAKE_INSTALL_BINDIR:PATH=${@os.path.relpath(d.getVar('bindir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_SBINDIR:PATH=${@os.path.relpath(d.getVar('sbindir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_LIBEXECDIR:PATH=${@os.path.relpath(d.getVar('libexecdir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_SYSCONFDIR:PATH=${sysconfdir} \ - -DCMAKE_INSTALL_SHAREDSTATEDIR:PATH=${@os.path.relpath(d.getVar('sharedstatedir'), d. getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_LOCALSTATEDIR:PATH=${localstatedir} \ - -DCMAKE_INSTALL_LIBDIR:PATH=${@os.path.relpath(d.getVar('libdir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_INCLUDEDIR:PATH=${@os.path.relpath(d.getVar('includedir'), d.getVar('prefix') + '/')} \ - -DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'), d.getVar('prefix') + '/')} \ - -DPYTHON_EXECUTABLE:PATH=${PYTHON} \ - -DPython_EXECUTABLE:PATH=${PYTHON} \ - -DPython3_EXECUTABLE:PATH=${PYTHON} \ - -DLIB_SUFFIX=${@d.getVar('baselib').replace('lib', '')} \ - -DCMAKE_INSTALL_SO_NO_EXE=0 \ - -DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \ - -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \ - -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON \ - -DFETCHCONTENT_FULLY_DISCONNECTED=ON \ + ${OECMAKE_ARGS} \ ${EXTRA_OECMAKE} \ -Wno-dev } From patchwork Mon Aug 21 16:07:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84EC4EE4993 for ; Mon, 21 Aug 2023 16:11:47 +0000 (UTC) Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) by mx.groups.io with SMTP id smtpd.web11.1047.1692634306704460078 for ; Mon, 21 Aug 2023 09:11:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=fYr46hnn; spf=pass (domain: gmail.com, ip: 209.85.208.46, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-52889bc61b6so4512679a12.0 for ; Mon, 21 Aug 2023 09:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634305; x=1693239105; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yi4WNTZvdI78aiIZIn+4I6MOZzcT6449uApeCStXHmA=; b=fYr46hnn9PT9Oqyy/cARxdPbCd2/nhEB33HxbgoKZyYlzzq6b0AlNwX3aEwDPAw7p/ vH4e1JiUh7v4EtUpO0FCe1TcGtBpNQaVV6BCaEwiYn7CxmI/g4XPGWdSqvB6IPfAnXV6 zKkEwUT3zTgTLUIPSKRIebCPtGP7JCgbe7vhOEb4w6QL1goqKkHz3tkvik71RCv9mT5J 1ZnFMSv2HYEH8Lw2wWJnu4T4yyBEQ5GxwvDgwqNHwAR4CBk/gxowXh0vSsXDafFACVNb a5FjdYwKRqr3Cstovsjj/8fBnxhaulNcrt+VVkNyYzelGJCugKLIHVYeid3l3FPfFBCW THxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634305; x=1693239105; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yi4WNTZvdI78aiIZIn+4I6MOZzcT6449uApeCStXHmA=; b=e7smD1t905F7Qt9a//wUVOX9/tUOIGHMuQS3LNA/qCDB2lubj/FONOheMU4UjTn4Jt WweERlnJcc8fmXTEZVxjb3hdSGcRLKk+8I6MjgqXPd4VJKeoqWz842rCgkHQQ9xi9NYI rj7hu8i62FAixYIfuyItKMxqZ1yz6cSU5/6zp6FW+Puf/dt+mkyBaWZjzObuV2L8kCRm w/WCFO8IQhg1zbogZK13sDVO0pNOrOX6wAeKidY7Cd07UNuAu5+n89s2FBqcooIoJtWK 1cFdy2q84vD5fm0WTfXNHgR3HgpdLkhxxKpS7NU0F+zDYkYu9uW4l5zp//CY1Z7duyB3 xM3w== X-Gm-Message-State: AOJu0YwpMVlUNN++x6tzLe1GLuoW0CZkw2dACDbeE/hEhY7rR5R72AJz Ezs3Ai1NpwB+YNeJ0k7P6IPOscW5rB8= X-Google-Smtp-Source: AGHT+IGLpYw8EGMpqa6G7p7IxJyeN0nkdsKCfwxx5P9uf4AccoJDwBjfltTThsp/MUWuTtL6o2e2tQ== X-Received: by 2002:aa7:d5ce:0:b0:523:69bf:2bd3 with SMTP id d14-20020aa7d5ce000000b0052369bf2bd3mr5195867eds.13.1692634304716; Mon, 21 Aug 2023 09:11:44 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:44 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 04/14] cmake.bbclass: cleanup spaces and tabs Date: Mon, 21 Aug 2023 18:07:07 +0200 Message-ID: <20230821161120.171956-5-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:47 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186467 Signed-off-by: Adrian Freihofer --- meta/classes-recipe/cmake.bbclass | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index 7c40df4cdbf..c63b0a80b18 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -90,12 +90,14 @@ def map_host_arch_to_uname_arch(host_arch): return "ppc64" return host_arch + cmake_do_generate_toolchain_file() { if [ "${BUILD_SYS}" = "${HOST_SYS}" ]; then cmake_crosscompiling="set( CMAKE_CROSSCOMPILING FALSE )" - else - cmake_sysroot="set( CMAKE_SYSROOT \"${RECIPE_SYSROOT}\" )" + else + cmake_sysroot="set( CMAKE_SYSROOT \"${RECIPE_SYSROOT}\" )" fi + cat > ${WORKDIR}/toolchain.cmake < X-Patchwork-Id: 29235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73E93EE49AA for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by mx.groups.io with SMTP id smtpd.web10.1055.1692634307134454338 for ; Mon, 21 Aug 2023 09:11:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=lqsi5w2u; spf=pass (domain: gmail.com, ip: 209.85.208.42, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-52a069edca6so1789916a12.3 for ; Mon, 21 Aug 2023 09:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634305; x=1693239105; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EpKqPHwYxKAzegLAS3aWmgTgfZUPe4SjRDBu36GJbTs=; b=lqsi5w2uLwGX2jZtO17lamac03+Wpg35/3tguqCm5sBsUD6DsFiT8AT5L2lg7+8NPH uZf25hwRL3uRwigbeYKuLaFlemi7M9xf7i1PXqr8jR6aL6vycWt+MgwVlI5gPV86yHC8 ekk4w9xnJVSDqDcBwSwBmkBNTe3iGFVwWn6jgc4DtP1D81hYtH+X9WCVIlwN2ybacvpT 00aM8Ttp8GcHDSHx2rjxCT1B2MXV6UidfjCsuUZUpzAoMbwbjdlT81tr6Ey7oPxNQywv OaZdGV4/X3Pr0Cfrwmdvz1dMTdL7KV2e7kDFdb8V8H5QR5sPltcbCwS8urnlaxUlMjkU OQjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634305; x=1693239105; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EpKqPHwYxKAzegLAS3aWmgTgfZUPe4SjRDBu36GJbTs=; b=aXfcJh+y5+kxcpqy0WIVkYtcYddiiSFjIXZ2RsmvKjImfKANvRwCt9lLkrywhSd01q DWYffKs59SYFxSchDA/gb+5hdKzJX1MBCGI/Z1mIAqVRaFszlz87kXGj/h1wihBR7iNo O/auV3quDnP2lyS0V0PdK2lFxOGimlFUtHWjHek5c0dQBOeJ7LJnxKKsl6Ibjs4JCcr3 kUQ/wcT2IW8m7iMzk0T1gaUOGK9O6oLYs3MXzNTkpHRqKD48NTsxBxJS0zTDOu1FQ95T v4Z+BZpivHHTLey6+FcblhyI/X4p4d07broq0ORdZiXBzVEDjC+xAY0iOObylAb8KN4q trag== X-Gm-Message-State: AOJu0YzN2NdoeATTESeHwsMdV4QbTpCwJXItGcN6XFxPgVjLtZpednvy NKgGt2+wC/SX1XlBKuTBF0NE+kyBYBY= X-Google-Smtp-Source: AGHT+IGq1G7TSSmxal+L4Al0zheeYLPSZLA67woAh+Rz2yCNcPqNb+WWVB2tXOiY4z+xaoIa8e5sCA== X-Received: by 2002:aa7:d058:0:b0:523:406a:5f6 with SMTP id n24-20020aa7d058000000b00523406a05f6mr5357235edo.12.1692634305409; Mon, 21 Aug 2023 09:11:45 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:45 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 05/14] cmake.bbclass: support qemu Date: Mon, 21 Aug 2023 18:07:08 +0200 Message-ID: <20230821161120.171956-6-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186468 Define the CMAKE_CROSSCOMPILING_EMULATOR variable similar to what the meson bbclass does. This allows for example to execute cross compilied unit tests on the build machine. CMAKE_CROSSCOMPILING_EMULATOR is a semi colon separated list of paramters which could directly handle the -L and the -E parameters. Creating a wrapper script is not absolutely mandatory. But anyway lets do it similar to what the meson.bbclass does and also disable pseudo. Signed-off-by: Adrian Freihofer --- meta/classes-recipe/cmake.bbclass | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index c63b0a80b18..944b0d8fefd 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -4,6 +4,13 @@ # SPDX-License-Identifier: MIT # +inherit qemu + +EXEWRAPPER_ENABLED:class-native = "False" +EXEWRAPPER_ENABLED:class-nativesdk = "False" +EXEWRAPPER_ENABLED ?= "${@bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', 'True', 'False', d)}" +DEPENDS:append = "${@' qemu-native' if d.getVar('EXEWRAPPER_ENABLED') == 'True' else ''}" + # Path to the CMake file to process. OECMAKE_SOURCEPATH ??= "${S}" @@ -156,6 +163,20 @@ EOF addtask generate_toolchain_file after do_patch before do_configure +cmake_do_generate_toolchain_file:append:class-target() { + if [ "${EXEWRAPPER_ENABLED}" = "True" ]; then + # Write out a qemu wrapper that will be used as exe_wrapper so that camake + # can run target helper binaries through that. This also allows to execute ctest. + qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_HOST}', ['${STAGING_DIR_HOST}/${libdir}','${STAGING_DIR_HOST}/${base_libdir}'])}" + echo "#!/bin/sh" > "${WORKDIR}/cmake-qemuwrapper" + echo "$qemu_binary \"\$@\"" >> "${WORKDIR}/cmake-qemuwrapper" + chmod +x "${WORKDIR}/cmake-qemuwrapper" + echo "set( CMAKE_CROSSCOMPILING_EMULATOR ${WORKDIR}/cmake-qemuwrapper)" \ + >> ${WORKDIR}/toolchain.cmake + fi +} + + CONFIGURE_FILES = "CMakeLists.txt" do_configure[cleandirs] = "${@d.getVar('B') if d.getVar('S') != d.getVar('B') else ''}" From patchwork Mon Aug 21 16:07:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29244 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C149EE49B1 for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by mx.groups.io with SMTP id smtpd.web11.1048.1692634308916394567 for ; Mon, 21 Aug 2023 09:11:49 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=dKHAKH8l; spf=pass (domain: gmail.com, ip: 209.85.167.48, mailfrom: adrian.freihofer@gmail.com) Received: by mail-lf1-f48.google.com with SMTP id 2adb3069b0e04-4ff933f9ca8so5224603e87.1 for ; Mon, 21 Aug 2023 09:11:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634307; x=1693239107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PHzrRL9KE6xqK1RuNlZlTqyLWTh2iPnU+0Wu1Uegqwo=; b=dKHAKH8lExTEWUsqaSu8pXlZi+2QvyxrZVligT+o4Z6DvYCn5EbkEDyCDhXuC4Lnyr 1aE0PfL10l/2kPw+KCbAaKJik9S89kVo5vn0ionBmnUW+WTXs/eHZ/lA7l0/wN22uHpE vuxxrYc6dZ3hcIdmm2a8vxuTMl0HsY74ZUmkjwxhM6Yp1Fwk9X3nFnU/7SuyG0fiv0Ko LsaljBLzuayh9RGiMNmN2eE1iK/OGwnTxA+mIKdQub0/MN1QmBP68gJIAx+z4Q83x0Yh OeJuwhUNVkvZePEF52+Et8M07XzVnqMgBB0KJuvEgQOQ1ATS2/Q2S6adiQ9W+Y74FugO VugQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634307; x=1693239107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PHzrRL9KE6xqK1RuNlZlTqyLWTh2iPnU+0Wu1Uegqwo=; b=ImTRE+FLyyn0bEZkZuNwgWdc6bajwcD4XUVGAKBYBhhVCHCl6A/11q2Ujjs9nfRKm8 GJifp5d3H3vUk55en89xZY15kd7dXAeRWlCjPbEhG/iM+XJcNZxoSywzoWu+izx7Kpyl poIdFIAl+S07Z1DxWB05BjH4SLIO7hhH7MwjDgjldgzFIiEraEVyxcfoaEZ2cf7tFrav Jd36ewucC9fh4zRsOe9J7U7AuCXyoFa78aIMcmCZiiMRjg3/gn4cfTWCXtWDngG9jAWL 4RFFq5dg4Ny2jJ2wqn0RXnGODETHo08GLWgK+MJzD1kkoKkE3iognqGD9qqL7UDHTnKI tvXg== X-Gm-Message-State: AOJu0YwpQSkbVxOPMJ4xPYZ3kdlpDo21YDHf3StrlYbY6Q2Q+V6HhDiR gKsOj12edTYkw2gRg4KoITQqk2bLrXM= X-Google-Smtp-Source: AGHT+IFIOp+lyfEBOwy+/4Df3Ljk7zzZXmNSG/Eba2JVWSsMAmLoXKkKk2SOZVr6x9RtQgAOjFsozg== X-Received: by 2002:ac2:4c92:0:b0:4fa:21d4:b3ca with SMTP id d18-20020ac24c92000000b004fa21d4b3camr4375643lfl.2.1692634306319; Mon, 21 Aug 2023 09:11:46 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:45 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 06/14] devtool: new ide plugin Date: Mon, 21 Aug 2023 18:07:09 +0200 Message-ID: <20230821161120.171956-7-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186470 Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/ide.py | 1190 ++++++++++++++++++++++++++++++++++++ 1 file changed, 1190 insertions(+) create mode 100755 scripts/lib/devtool/ide.py diff --git a/scripts/lib/devtool/ide.py b/scripts/lib/devtool/ide.py new file mode 100755 index 00000000000..7c7040232c4 --- /dev/null +++ b/scripts/lib/devtool/ide.py @@ -0,0 +1,1190 @@ +#! /usr/bin/env python3 +# +# Copyright (C) 2023 Siemens AG +# +# SPDX-License-Identifier: GPL-2.0-only +# + +"""Devtool ide plugin""" + +import os +import stat +import logging +import json +import re +import shutil +from argparse import RawTextHelpFormatter +from enum import IntEnum, auto + +import bb +from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, DevtoolError, parse_recipe +from devtool.standard import get_real_srctree + +SHARED_SYSROOT_RECIPES = ['none', 'meta-ide-support', 'build-sysroots'] +SUPPORTED_IDES = ['code', 'none'] + +logger = logging.getLogger('devtool') + + +class TargetDevice: + """SSH remote login parameters""" + + def __init__(self, args): + self.extraoptions = '' + if args.no_host_check: + self.extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' + self.ssh_sshexec = 'ssh' + if args.ssh_exec: + self.ssh_sshexec = args.ssh_exec + self.ssh_port = '' + if args.port: + self.ssh_port = "-p %s" % args.port + if args.key: + self.extraoptions += ' -i %s' % args.key + + self.target = args.target + target_sp = args.target.split('@') + if len(target_sp) == 1: + self.login = "" + self.host = target_sp[0] + elif len(target_sp) == 2: + self.login = target_sp[0] + self.host = target_sp[1] + else: + logger.error("Invalid target argument: %s" % args.target) + + +class RecipeNative: + def __init__(self, name, target_arch=None): + self.name = name + self.target_arch = target_arch + self.bootstrap_tasks = [self.name + ':do_addto_recipe_sysroot'] + self.staging_bindir_native = None + self.target_sys = None + self.__native_bin = None + + def initialize(self, config, workspace, tinfoil): + recipe_d = parse_recipe( + config, tinfoil, self.name, appends=True, filter_workspace=False) + if not recipe_d: + raise DevtoolError("Parsing %s recipe failed" % self.name) + self.staging_bindir_native = os.path.realpath( + recipe_d.getVar('STAGING_BINDIR_NATIVE')) + self.target_sys = recipe_d.getVar('TARGET_SYS') + + @property + def native_bin(self): + if not self.__native_bin: + raise DevtoolError("native binary name is not defined.") + return self.__native_bin + + +class RecipeGdbCross(RecipeNative): + def __init__(self, args, target_arch, target_device, gdbserver_multi=True): + super().__init__('gdb-cross-' + target_arch, target_arch) + self.target_device = target_device + self.gdb = None + self.gdbserver_port_next = int(args.gdbserver_port_start) + self.gdbserver_multi = gdbserver_multi + self.config_db = {} + + def initialize(self, config, workspace, tinfoil): + super().initialize(config, workspace, tinfoil) + gdb_bin = self.target_sys + '-gdb' + gdb_path = os.path.join( + self.staging_bindir_native, self.target_sys, gdb_bin) + self.gdb = gdb_path + + @property + def host(self): + return self.target_device.host + + def __gdbserver_start_cmd(self, binary, port): + if self.gdbserver_multi: + gdbserver_cmd = "/usr/bin/gdbserver --multi :%s" % ( + port) + else: + gdbserver_cmd = "/usr/bin/gdbserver --once :%s %s" % ( + port, binary) + return "%s %s %s %s 'sh -c \"%s\"'" % ( + self.target_device.ssh_sshexec, self.target_device.ssh_port, self.target_device.extraoptions, self.target_device.target, gdbserver_cmd) + + def setup_gdbserver_config(self, binary, script_dir): + if binary in self.config_db: + raise DevtoolError( + "gdbserver config for binary %s is already generated" % binary) + + port = self.gdbserver_port_next + self.gdbserver_port_next += 1 + config_entry = { + "port": port, + } + if script_dir: + cmd_lines = ['#!/bin/sh'] + cmd_lines.append(self.__gdbserver_start_cmd(binary, port)) + binary_name_pretty = binary.replace(os.sep, '-') + start_script_name = 'gdbserver_start_%d_%s' % ( + port, binary_name_pretty) + if self.gdbserver_multi: + start_script_name += "_m" + start_script_file = os.path.join(script_dir, start_script_name) + config_entry['gdbserver_start_script'] = start_script_file + config_entry['pretty_id'] = 'gdbserver start %d %s' % ( + port, binary) + + bb.utils.mkdirhier(script_dir) + with open(start_script_file, 'w') as script_f: + script_f.write(os.linesep.join(cmd_lines)) + script_f.write(os.linesep) + st = os.stat(start_script_file) + os.chmod(start_script_file, st.st_mode | stat.S_IEXEC) + + self.config_db[binary] = config_entry + return config_entry + + def get_gdbserver_start_scripts(self): + for conf in self.config_db.values(): + yield (conf['pretty_id'], conf['gdbserver_start_script']) + + def get_gdbserver_pretty_id(self, binary): + return self.config_db[binary]['pretty_id'] + + def get_gdbserver_port(self, binary): + return self.config_db[binary]['port'] + + def get_gdbserver_start_script(self, binary): + return self.config_db[binary]['gdbserver_start_script'] + + +class RecipeImage: + """Handle some image recipe related properties""" + + def __init__(self, name): + self.combine_dbg_image = False + self.debuginfod = False + self.name = name + self.package_debug_split_style = None + self.rootfs = None + self.__rootfs_dbg = None + self.bootstrap_tasks = [self.name + ':do_build'] + + def initialize(self, config, tinfoil): + image_d = parse_recipe( + config, tinfoil, self.name, appends=True, filter_workspace=False) + if not image_d: + raise DevtoolError( + "Parsing image recipe %s failed" % self.name) + if 'debuginfod' in image_d.getVar('DISTRO_FEATURES').split(): + # image_config.debuginfod = True + logger.warning("Support for debuginfod is not implemented yet.") + + self.package_debug_split_style = image_d.getVar( + 'PACKAGE_DEBUG_SPLIT_STYLE') + + workdir = image_d.getVar('WORKDIR') + self.rootfs = os.path.join(workdir, 'rootfs') + if image_d.getVar('IMAGE_GEN_DEBUGFS') == "1": + self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg') + + self.combine_dbg_image = bb.data.inherits_class( + 'image-combined-dbg', image_d) + + @property + def debug_support(self): + return bool(self.debuginfod or self.rootfs_dbg) + + @property + def rootfs_dbg(self): + if self.__rootfs_dbg and os.path.isdir(self.__rootfs_dbg): + return self.__rootfs_dbg + return None + + def solib_search_path_rootfs(self): + """Search for folders with shared libraries in the rootfs and rootfs-dbg + + This is based on the assumption that the PACKAGE_DEBUG_SPLIT_STYLE variable from the image + is the global setting which is used by most packages. Even if this variable does not seem + to make sense in the image context. + """ + rootfs_solib_search_path = [] + rootfs_dbg_solib_search_path = [] + if self.package_debug_split_style in ['debug-with-srcpkg', '.debug']: + if self.combine_dbg_image: + rootfs_dbg_solib_search_path = [ + "/lib", "/lib/.debug", "/usr/lib", "/usr/lib/.debug"] + else: + logger.warn( + 'Adding IMAGE_CLASSES += "image-combined-dbg" offers better remote debugging experience.') + rootfs_solib_search_path = [ + "/lib", "/usr/lib"] + rootfs_dbg_solib_search_path = [ + "/lib/.debug", "/usr/lib/.debug"] + elif self.package_debug_split_style == 'debug-file-directory': + rootfs_dbg_solib_search_path = ["/usr/lib/debug"] + else: + logger.warning( + "Cannot find solib search path for a rootfs built with PACKAGE_DEBUG_SPLIT_STYLE=%s." % self.package_debug_split_style) + + sym_dirs = [] + for dbgdir in rootfs_solib_search_path: + sym_dirs.append(os.path.join( + self.rootfs, dbgdir.lstrip('/'))) + for dbgdir in rootfs_dbg_solib_search_path: + sym_dirs.append(os.path.join( + self.rootfs_dbg, dbgdir.lstrip('/'))) + + return sym_dirs + + +class RecipeMetaIdeSupport: + """Handle some meta-ide-support recipe related properties""" + + def __init__(self): + self.bootstrap_tasks = ['meta-ide-support:do_build'] + self.topdir = None + self.datadir = None + self.deploy_dir_image = None + self.build_sys = None + # From toolchain-scripts + self.real_multimach_target_sys = None + + def initialize(self, config, tinfoil): + meta_ide_support_d = parse_recipe( + config, tinfoil, 'meta-ide-support', appends=True, filter_workspace=False) + if not meta_ide_support_d: + raise DevtoolError("Parsing meta-ide-support recipe failed") + + self.topdir = meta_ide_support_d.getVar('TOPDIR') + self.datadir = meta_ide_support_d.getVar('datadir') + self.deploy_dir_image = meta_ide_support_d.getVar( + 'DEPLOY_DIR_IMAGE') + self.build_sys = meta_ide_support_d.getVar('BUILD_SYS') + self.real_multimach_target_sys = meta_ide_support_d.getVar( + 'REAL_MULTIMACH_TARGET_SYS') + + +class RecipeBuildSysroots: + def __init__(self): + self.standalone_sysroot = None + self.standalone_sysroot_native = None + self.bootstrap_tasks = ['build-sysroots:do_build'] + + def initialize(self, config, tinfoil): + build_sysroots_d = parse_recipe( + config, tinfoil, 'build-sysroots', appends=True, filter_workspace=False) + if not build_sysroots_d: + raise DevtoolError("Parsing build-sysroots recipe failed") + self.standalone_sysroot = build_sysroots_d.getVar( + 'STANDALONE_SYSROOT') + self.standalone_sysroot_native = build_sysroots_d.getVar( + 'STANDALONE_SYSROOT_NATIVE') + + +class SharedSysrootsEnv: + def __init__(self): + self.ide_support = None + self.build_sysroots = None + + def initialize(self, ide_support, build_sysroots): + self.ide_support = ide_support + self.build_sysroots = build_sysroots + + def __vscode_update_kits(self): + """Expose the toolchain of the dSDK""" + datadir = self.ide_support.datadir + deploy_dir_image = self.ide_support.deploy_dir_image + real_multimach_target_sys = self.ide_support.real_multimach_target_sys + standalone_sysroot_native = self.build_sysroots.standalone_sysroot_native + vscode_ws_path = os.path.join( + os.environ['HOME'], '.local', 'share', 'CMakeTools') + cmake_kits_path = os.path.join(vscode_ws_path, 'cmake-tools-kits.json') + oecmake_generator = "Ninja" + env_script = os.path.join( + deploy_dir_image, 'environment-setup-' + real_multimach_target_sys) + + logger.info("updating %s" % cmake_kits_path) + + if not os.path.isdir(vscode_ws_path): + os.makedirs(vscode_ws_path) + cmake_kits_old = [] + if os.path.exists(cmake_kits_path): + with open(cmake_kits_path, 'r', encoding='utf-8') as cmake_kits_file: + cmake_kits_old = json.load(cmake_kits_file) + cmake_kits = cmake_kits_old.copy() + + cmake_kit_new = { + "name": "OE " + real_multimach_target_sys, + "environmentSetupScript": env_script, + "toolchainFile": standalone_sysroot_native + datadir + "/cmake/OEToolchainConfig.cmake", + "preferredGenerator": { + "name": oecmake_generator + } + } + + def merge_kit(cmake_kits, cmake_kit_new): + i = 0 + while i < len(cmake_kits): + if 'environmentSetupScript' in cmake_kits[i] and \ + cmake_kits[i]['environmentSetupScript'] == cmake_kit_new['environmentSetupScript']: + cmake_kits[i] = cmake_kit_new + return + i += 1 + cmake_kits.append(cmake_kit_new) + merge_kit(cmake_kits, cmake_kit_new) + + if cmake_kits != cmake_kits_old: + bb.note("Updating: %s" % cmake_kits_path) + with open(cmake_kits_path, 'w', encoding='utf-8') as cmake_kits_file: + json.dump(cmake_kits, cmake_kits_file, indent=4) + + def setup_ide(self, args): + if args.ide == 'code': + self.__vscode_update_kits() + + +class BuildTool(IntEnum): + UNDEFINED = auto() + CMAKE = auto() + MESON = auto() + + +class RecipeModified: + + def __init__(self, name): + self.name = name + self.bootstrap_tasks = [name + ':do_install'] + # workspace + self.real_srctree = None + self.srctree = None + self.temp_dir = None + self.bbappend = None + # recipe variables from d.getVar + self.b = None + self.bpn = None + self.d = None + self.fakerootcmd = None + self.fakerootenv = None + self.package_arch = None + self.path = None + self.recipe_sysroot = None + self.recipe_sysroot_native = None + self.staging_incdir = None + self.target_arch = None + self.workdir = None + # replicate bitbake build environment + self.__exported_vars = None + self.cmd_compile = None + # main build tool used by this recipe + self.build_tool = BuildTool.UNDEFINED + # build_tool = cmake + self.oecmake_generator = None + self.__cmake_cache_vars = None + # build_tool = meson + self.meson_buildtype = None + self.meson_wrapper = None + self.mesonopts = None + self.extra_oemeson = None + self.meson_cross_file = None + # vscode + self.dot_code_dir = None + # TODO: remove calling devtool + self.bb_env_passthrough_additions = None + self.bbpath = None + self.bitbakepath = None + self.topdir = None + + def initialize(self, config, workspace, tinfoil): + recipe_d = parse_recipe( + config, tinfoil, self.name, appends=True, filter_workspace=False) + if not recipe_d: + raise DevtoolError("Parsing %s recipe failed" % self.name) + + # Verify this recipe is built as externalsrc setup by devtool modify + workspacepn = check_workspace_recipe( + workspace, self.name, bbclassextend=True) + self.srctree = workspace[workspacepn]['srctree'] + # Need to grab this here in case the source is within a subdirectory + self.real_srctree = get_real_srctree( + self.srctree, recipe_d.getVar('S'), recipe_d.getVar('WORKDIR')) + self.bbappend = workspace[workspacepn]['bbappend'] + + self.temp_dir = os.path.join(config.workspace_path, 'temp', self.name) + if os.path.exists(self.temp_dir): + shutil.rmtree(self.temp_dir) + + self.b = recipe_d.getVar('B') + self.bpn = recipe_d.getVar('BPN') + self.d = recipe_d.getVar('D') + self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD') + self.fakerootenv = recipe_d.getVar('FAKEROOTENV') + self.package_arch = recipe_d.getVar('PACKAGE_ARCH') + self.path = recipe_d.getVar('PATH') + self.recipe_sysroot = os.path.realpath( + recipe_d.getVar('RECIPE_SYSROOT')) + self.recipe_sysroot_native = os.path.realpath( + recipe_d.getVar('RECIPE_SYSROOT_NATIVE')) + self.staging_incdir = os.path.realpath( + recipe_d.getVar('STAGING_INCDIR')) + self.target_arch = recipe_d.getVar('TARGET_ARCH') + self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR')) + + self.bb_env_passthrough_additions = recipe_d.getVar( + 'BB_ENV_PASSTHROUGH_ADDITIONS') + self.bbpath = recipe_d.getVar('BBPATH') + self.bitbakepath = recipe_d.getVar('BITBAKEPATH') + self.topdir = recipe_d.getVar('TOPDIR') + + self.__init_exported_variables(recipe_d) + + if bb.data.inherits_class('cmake', recipe_d): + self.oecmake_generator = recipe_d.getVar('OECMAKE_GENERATOR') + self.__init_cmake_preset_cache(recipe_d) + self.build_tool = BuildTool.CMAKE + elif bb.data.inherits_class('meson', recipe_d): + self.meson_buildtype = recipe_d.getVar('MESON_BUILDTYPE') + self.mesonopts = recipe_d.getVar('MESONOPTS') + self.extra_oemeson = recipe_d.getVar('EXTRA_OEMESON') + self.meson_cross_file = recipe_d.getVar('MESON_CROSS_FILE') + self.build_tool = BuildTool.MESON + + self.dot_code_dir = os.path.join(self.srctree, '.vscode') + + def append_to_bbappend(self, append_text): + with open(self.bbappend, 'a') as bbap: + bbap.write(append_text) + + def remove_from_bbappend(self, append_text): + with open(self.bbappend, 'r') as bbap: + text = bbap.read() + new_text = text.replace(append_text, '') + with open(self.bbappend, 'w') as bbap: + bbap.write(new_text) + + def debug_build_config(self, args): + """Explicitely set for example CMAKE_BUILD_TYPE to Debug if not defined otherwise""" + if self.build_tool == BuildTool.CMAKE: + append_text = os.linesep + \ + 'OECMAKE_ARGS:append = " -DCMAKE_BUILD_TYPE:STRING=Debug"' + os.linesep + if args.debug_build_config and not 'CMAKE_BUILD_TYPE' in self.__cmake_cache_vars: + self.__cmake_cache_vars['CMAKE_BUILD_TYPE'] = { + "type": "STRING", + "value": "Debug", + } + self.append_to_bbappend(append_text) + elif 'CMAKE_BUILD_TYPE' in self.__cmake_cache_vars: + del self.__cmake_cache_vars['CMAKE_BUILD_TYPE'] + self.remove_from_bbappend(append_text) + elif self.build_tool == BuildTool.MESON: + append_text = os.linesep + 'MESON_BUILDTYPE = "debug"' + os.linesep + if args.debug_build_config and self.meson_buildtype != "debug": + self.mesonopts.replace( + '--buildtype ' + self.meson_buildtype, '--buildtype debug') + self.append_to_bbappend(append_text) + elif self.meson_buildtype == "debug": + self.mesonopts.replace( + '--buildtype debug', '--buildtype plain') + self.remove_from_bbappend(append_text) + elif args.debug_build_config: + logger.warn( + "--debug-build-config is not implemented for this build tool yet.") + + def solib_search_path_sysroot(self): + return [os.path.join(self.recipe_sysroot, p) for p in ['lib', 'usr/lib']] + + def solib_search_path(self, image): + return image.solib_search_path_rootfs() + self.solib_search_path_sysroot() + + def solib_search_path_str(self, image): + return ':'.join(self.solib_search_path(image)) + + def __init_exported_variables(self, d): + """Find all variables with export flag set.""" + exported_vars = {} + + vars = (key for key in d.keys() if not key.startswith( + "__") and not d.getVarFlag(key, "func", False)) + for var in vars: + func = d.getVarFlag(var, "func", False) + if d.getVarFlag(var, 'python', False) and func: + continue + export = d.getVarFlag(var, "export", False) + unexport = d.getVarFlag(var, "unexport", False) + if not export and not unexport and not func: + continue + if unexport: + continue + + val = d.getVar(var) + if val is None: + continue + if set(var) & set("-.{}+"): + logger.warn( + "Warning: Found invalid character in variable name %s", str(var)) + continue + varExpanded = d.expand(var) + val = str(val) + + if varExpanded.startswith("BASH_FUNC_"): + varExpanded = varExpanded[10:-2] + val = val[3:] # Strip off "() " + logger.warn("Warning: BASH_FUNC_ is not exported to cmake presets (%s() %s)" % ( + varExpanded, val)) + continue + + if func: + code_line = "line: {0}, file: {1}\n".format( + d.getVarFlag(var, "lineno", False), + d.getVarFlag(var, "filename", False)) + val = val.rstrip('\n') + logger.warn("Warning: exported shell function %s() is not exported (%s)" % + (varExpanded, code_line)) + continue + + if export: + exported_vars[varExpanded] = val.strip() + continue + + self.__exported_vars = exported_vars + + def __init_cmake_preset_cache(self, d): + """Replicate the cmake configure arguments with all details to share on build folder between bitbake and SDK.""" + site_file = os.path.join(self.workdir, 'site-file.cmake') + if os.path.exists(site_file): + print("Warning: site-file.cmake is not supported") + + cache_vars = {} + oecmake_args = d.getVar('OECMAKE_ARGS').split() + extra_oecmake = d.getVar('EXTRA_OECMAKE').split() + for param in oecmake_args + extra_oecmake: + d_pref = "-D" + if param.startswith(d_pref): + param = param[len(d_pref):] + else: + print("Error: expected a -D") + param_s = param.split('=', 1) + param_nt = param_s[0].split(':', 1) + + def handle_undefined_variable(var): + if var.startswith('${') and var.endswith('}'): + return '' + else: + return var + # Example: FOO=ON + if len(param_nt) == 1: + cache_vars[param_s[0]] = handle_undefined_variable(param_s[1]) + # Example: FOO:PATH=/tmp + elif len(param_nt) == 2: + cache_vars[param_nt[0]] = { + "type": param_nt[1], + "value": handle_undefined_variable(param_s[1]), + } + else: + print("Error: cannot parse %s" % param) + self.__cmake_cache_vars = cache_vars + + def __cmake_preset(self): + toolchain_file = os.path.join(self.workdir, 'toolchain.cmake') + preset_name = self.bpn + "-" + self.package_arch + preset_display_name = self.bpn + ": " + self.package_arch + cmake_executable = os.path.join( + self.recipe_sysroot_native, 'usr', 'bin', 'cmake') + self.cmd_compile = cmake_executable + " --build --preset " + preset_name + + preset_dict_configure = { + "name": preset_name, + "displayName": preset_display_name, + "description": "Bitbake build environment for the recipe %s compiled for %s" % (self.bpn, self.package_arch), + "binaryDir": self.b, + "generator": self.oecmake_generator, + "toolchainFile": toolchain_file, + "cacheVariables": self.__cmake_cache_vars, + "environment": self.__exported_vars, + "cmakeExecutable": cmake_executable + } + + preset_dict_build = { + "name": preset_name, + "displayName": preset_display_name, + "description": "Bitbake build environment for the recipe %s compiled for %s" % (self.bpn, self.package_arch), + "configurePreset": preset_name, + "inheritConfigureEnvironment": True + } + + preset_dict_test = { + "name": preset_name, + "displayName": preset_display_name, + "description": "Bitbake build environment for the recipe %s compiled for %s" % (self.bpn, self.package_arch), + "configurePreset": preset_name, + "inheritConfigureEnvironment": True + } + + preset_dict = { + "version": 3, # cmake 3.21, backward compatible with kirkstone + "configurePresets": [preset_dict_configure], + "buildPresets": [preset_dict_build], + "testPresets": [preset_dict_test] + } + + logger.info("generating cmake preset for recipe %s" % self.bpn) + preset_file = os.path.join(self.real_srctree, 'CMakeUserPresets.json') + with open(preset_file, 'w') as outfile: + json.dump(preset_dict, outfile, indent=4) + + @staticmethod + def update_json_file(dot_code_dir, json_file, update_dict): + json_path = os.path.join(dot_code_dir, json_file) + logger.info("Updating vscode %s (%s)" % (json_file, json_path)) + if not os.path.exists(dot_code_dir): + os.makedirs(dot_code_dir) + try: + with open(json_path) as f: + orig_dict = json.load(f) + except json.decoder.JSONDecodeError: + logger.info( + "Decoding %s failed. Probably because of comments in the json file" % json_path) + orig_dict = {} + except FileNotFoundError: + orig_dict = {} + orig_dict.update(update_dict) + with open(json_path, 'w') as f: + json.dump(orig_dict, f, indent=4) + + def __vscode_settings_cmake(self, settings_dict): + """Add cmake specific settings to settings.json. + + Note: most settings are passed to the cmake preset. + """ + if self.build_tool != BuildTool.CMAKE: + return + settings_dict["cmake.configureOnOpen"] = False + settings_dict["cmake.sourceDirectory"] = self.real_srctree + + def __gen_meson_wrapper(self): + """Generate a wrapper script to call meson with cross environment""" + bb.utils.mkdirhier(self.temp_dir) + meson_wrapper = os.path.join(self.temp_dir, 'meson') + meson_real = os.path.join( + self.recipe_sysroot_native, 'usr', 'bin', 'meson.real') + with open(meson_wrapper, 'w') as mwrap: + mwrap.write("#!/bin/sh" + os.linesep) + for var, val in self.__exported_vars.items(): + mwrap.write('export %s="%s"' % (var, val) + os.linesep) + mwrap.write("unset CC CXX CPP LD AR NM STRIP" + os.linesep) + private_temp = os.path.join(self.b, "meson-private", "tmp") + mwrap.write('mkdir -p "%s"' % private_temp + os.linesep) + mwrap.write('export TMPDIR="%s"' % private_temp + os.linesep) + mwrap.write('exec "%s" "$@"' % meson_real + os.linesep) + st = os.stat(meson_wrapper) + os.chmod(meson_wrapper, st.st_mode | stat.S_IEXEC) + self.meson_wrapper = meson_wrapper + self.cmd_compile = meson_wrapper + " compile -C " + self.b + + def __vscode_settings_meson(self, settings_dict): + if self.build_tool != BuildTool.MESON: + return + settings_dict["mesonbuild.mesonPath"] = self.meson_wrapper + + confopts = self.mesonopts.split() + confopts += self.meson_cross_file.split() + confopts += self.extra_oemeson.split() + settings_dict["mesonbuild.configureOptions"] = confopts + settings_dict["mesonbuild.buildFolder"] = self.b + + def vscode_settings(self): + files_excludes = { + "**/.git/**": True, + "**/oe-local-files/**": True, + "**/oe-logs/**": True, + "**/oe-workdir/**": True, + "**/source-date-epoch/**": True + } + python_exclude = [ + "**/.git/**", + "**/oe-local-files/**", + "**/oe-logs/**", + "**/oe-workdir/**", + "**/source-date-epoch/**" + ] + settings_dict = { + "files.watcherExclude": files_excludes, + "files.exclude": files_excludes, + "python.analysis.exclude": python_exclude + } + self.__vscode_settings_cmake(settings_dict) + self.__vscode_settings_meson(settings_dict) + + settings_file = 'settings.json' + RecipeModified.update_json_file( + self.dot_code_dir, settings_file, settings_dict) + + def __vscode_extensions_cmake(self, recommendations): + if self.build_tool != BuildTool.CMAKE: + return + recommendations += [ + "twxs.cmake", + "ms-vscode.cmake-tools", + "ms-vscode.cpptools", + "ms-vscode.cpptools-extension-pack", + "ms-vscode.cpptools-themes" + ] + + def __vscode_extensions_meson(self, recommendations): + if self.build_tool != BuildTool.MESON: + return + recommendations += [ + 'mesonbuild.mesonbuild' + ] + + def vscode_extensions(self): + recommendations = [ + "EditorConfig.EditorConfig" + ] + self.__vscode_extensions_cmake(recommendations) + self.__vscode_extensions_meson(recommendations) + extensions_file = 'extensions.json' + RecipeModified.update_json_file( + self.dot_code_dir, extensions_file, {"recommendations": recommendations}) + + def which(self, executable): + bin_path = shutil.which(executable, path=self.path) + if not bin_path: + raise DevtoolError( + 'Cannot find %s. Probably the recipe %s is not built yet.' % (executable, self.bpn)) + return bin_path + + @staticmethod + def vscode_intelli_sense_mode(compiler): + unknown = False + linux = '' + if 'linux' in compiler: + linux = "linux-" + + cxx = 'unknown-' + if 'g++' in compiler: + cxx = 'gcc-' + elif 'clang' in compiler: + cxx = 'clang-' + else: + unknown = True + + arch = 'unknown' + if 'aarch64' in compiler: + arch = 'arm64' + elif 'arm' in compiler: + arch = 'arm' + elif 'x86_64' in compiler: + arch = 'x64' + elif 'i386' in compiler: + arch = 'x86' + else: + unknown = True + + mode = linux + cxx + arch + if unknown: + logger.warn( + "Cannot guess intelliSenseMode for compiler %s (%s)" % (compiler, mode)) + return '${default}' + return mode + + def vscode_c_cpp_properties(self): + properties_dict = { + "name": "Yocto " + self.package_arch, + } + if self.build_tool == BuildTool.CMAKE: + properties_dict["configurationProvider"] = "ms-vscode.cmake-tools" + elif self.build_tool == BuildTool.MESON: + properties_dict["configurationProvider"] = "mesonbuild.mesonbuild" + + properties_dicts = { + "configurations": [ + properties_dict + ], + "version": 4 + } + prop_file = 'c_cpp_properties.json' + dot_code_dir = os.path.join(self.srctree, '.vscode') + RecipeModified.update_json_file( + dot_code_dir, prop_file, properties_dicts) + + def vscode_launch_cppdbg(self, image, gdb_cross, binary): + gdb_cross.setup_gdbserver_config(binary, self.temp_dir) + pretty_id = gdb_cross.get_gdbserver_pretty_id(binary) + gdbserver_port = gdb_cross.get_gdbserver_port(binary) + + launch_config = { + "name": pretty_id, + "type": "cppdbg", + "request": "launch", + "program": os.path.join(self.d, binary.lstrip('/')), + "stopAtEntry": True, + "cwd": "${workspaceFolder}", + "environment": [], + "externalConsole": False, + "MIMode": "gdb", + "preLaunchTask": pretty_id, + "miDebuggerPath": gdb_cross.gdb, + "miDebuggerServerAddress": "%s:%d" % (gdb_cross.host, gdbserver_port) + } + + # Search for header files in recipe-sysroot. + src_file_map = { + "/usr/include": os.path.join(self.recipe_sysroot, "usr", "include") + } + # First of all search for not stripped binaries in the image folder. + # These binaries are copied (and optionally stripped) by deploy-target + setup_commands = [ + { + "description": "sysroot", + "text": "set sysroot " + self.d + } + ] + + if image.rootfs_dbg: + launch_config['additionalSOLibSearchPath'] = self.solib_search_path_str( + image) + src_file_map["/usr/src/debug"] = os.path.join( + image.rootfs_dbg, "usr", "src", "debug") + else: + logger.warning( + "Cannot setup debug symbols configuration for GDB. IMAGE_GEN_DEBUGFS is not enabled.") + + launch_config['sourceFileMap'] = src_file_map + launch_config['setupCommands'] = setup_commands + return launch_config + + @staticmethod + def is_elf_file(file_path): + with open(file_path, "rb") as f: + data = f.read(4) + if data == b'\x7fELF': + return True + return False + + def find_installed_binaries(self): + """find all executable elf files in the image directory""" + binaries = [] + d_len = len(self.d) + re_so = re.compile('.*\.so[.0-9]*$') + for root, _, files in os.walk(self.d, followlinks=False): + for file in files: + if os.path.islink(file): + continue + if re_so.match(file): + continue + abs_name = os.path.join(root, file) + if os.access(abs_name, os.X_OK) and RecipeModified.is_elf_file(abs_name): + binaries.append(abs_name[d_len:]) + return binaries + + def vscode_launch(self, image, gdb_cross): + binaries = self.find_installed_binaries() + configurations = [self.vscode_launch_cppdbg( + image, gdb_cross, binary) for binary in binaries] + launch_dict = { + "version": "0.2.0", + "configurations": configurations + } + launch_file = 'launch.json' + RecipeModified.update_json_file( + self.dot_code_dir, launch_file, launch_dict) + + @staticmethod + def get_unique_gdbinit_name(binary): + return 'gdbinit' + binary.replace(os.sep, '-') + + def none_launch(self, image, gdb_cross): + """generate a gdbinit file per executable""" + binaries = self.find_installed_binaries() + for binary in binaries: + gdb_cross.setup_gdbserver_config(binary, self.temp_dir) + gdbserver_port = str(gdb_cross.get_gdbserver_port(binary)) + if gdb_cross.gdbserver_multi: + target_help = '# gdbserver --multi :' + gdbserver_port + remote_cmd = 'target extended-remote ' + else: + target_help = '# gdbserver :' + gdbserver_port + ' ' + binary + remote_cmd = 'target remote ' + gdbinit_path = os.path.join( + self.real_srctree, RecipeModified.get_unique_gdbinit_name(binary)) + + gdbinit_lines = ['# This file is generated by devtool ide'] + gdbinit_lines.append('# On the remote target:') + gdbinit_lines.append(target_help) + gdbinit_lines.append('# On the build machine:') + gdbinit_lines.append('# cd ' + self.real_srctree) + gdbinit_lines.append( + '# ' + gdb_cross.gdb + ' -ix ' + gdbinit_path) + + gdbinit_lines.append('set sysroot ' + self.d) + gdbinit_lines.append('set substitute-path "/usr/include" "' + + os.path.join(self.recipe_sysroot, 'usr', 'include') + '"') + if image.debuginfod: + gdbinit_lines.append('set debuginfod enabled on') + else: + gdbinit_lines.append('set debuginfod enabled off') + if image.rootfs_dbg: + gdbinit_lines.append( + 'set solib-search-path "' + self.solib_search_path_str(image) + '"') + gdbinit_lines.append('set substitute-path "/usr/src/debug" "' + os.path.join( + image.rootfs_dbg, 'usr', 'src', 'debug') + '"') + gdbinit_lines.append( + remote_cmd + gdb_cross.host + ':' + gdbserver_port) + gdbinit_lines.append('set remote exec-file ' + binary) + gdbinit_lines.append('run ' + os.path.join(self.d, binary)) + + with open(gdbinit_path, 'w') as gdbinit_file: + gdbinit_file.write('\n'.join(gdbinit_lines)) + + def gen_fakeroot_install_script(self): + """Run the run.do_install script from bitbake under pseudo so that it picks up the appropriate file permissions""" + cmd_lines = ['#!/bin/sh'] + # Ensure the do compile step gets always executed without pseuso before do install + if self.cmd_compile: + cmd_compile = "( cd %s && %s)" % ( + self.real_srctree, self.cmd_compile) + cmd_lines.append(cmd_compile) + if not os.access(self.fakerootcmd, os.X_OK): + raise DevtoolError( + "pseudo executable %s could not be found" % self.fakerootcmd) + run_do_install = os.path.join(self.workdir, 'temp', 'run.do_install') + + if not os.access(run_do_install, os.X_OK): + raise DevtoolError( + "run script does not exists: %s" % run_do_install) + + # Set up the appropriate environment + newenv = dict(os.environ) + for varvalue in self.fakerootenv.split(): + if '=' in varvalue: + splitval = varvalue.split('=', 1) + newenv[splitval[0]] = splitval[1] + + # Cleanup TMPDIR before calling do_install independently from bitbake + # This is anyway outdated after do_install has been executed. + # But maybe there should be a cleaner solution here. + rm_in_workdir = ' '.join([os.path.join(self.workdir, d) for d in [ + "package", "packages-split", "pkgdata", "sstate-install-package", "debugsources.list", "*.spec"]]) + install_cmd = '%s /bin/sh -c "rm -rf %s/* %s && %s"' % ( + self.fakerootcmd, self.d, rm_in_workdir, run_do_install) + for var, val in newenv.items(): + cmd_lines.append('export %s="%s"' % (var, val)) + cmd_lines.append(install_cmd) + return self.write_script(cmd_lines, 'bb_run_do_install') + + def write_script(self, cmd_lines, script_name): + bb.utils.mkdirhier(self.temp_dir) + script_file = os.path.join(self.temp_dir, script_name) + with open(script_file, 'w') as script_f: + script_f.write(os.linesep.join(cmd_lines)) + st = os.stat(script_file) + os.chmod(script_file, st.st_mode | stat.S_IEXEC) + return script_file + + def vscode_tasks(self, args, gdb_cross): + run_do_install = self.gen_fakeroot_install_script() + pythonpath_new = os.path.realpath( + os.path.join(self.bitbakepath, '../lib')) + try: + pythonpath = os.environ['PYTONPATH'].split(':') + if pythonpath_new not in pythonpath: + pythonpath = pythonpath_new + ":" + pythonpath + except KeyError: + pythonpath = pythonpath_new + install_label = "do_install %s" % self.bpn + deploy_label = "devtool deploy-target %s" % self.bpn + bb_env = { + # "BBPATH": self.bbpath, + "PYTHONPATH": pythonpath, + "BUILDDIR": self.topdir, + # Use e.g. Python from host, not python-native + "PATH": os.environ['PATH'], + "BB_ENV_PASSTHROUGH_ADDITIONS": self.bb_env_passthrough_additions + } + bb_options = { + "cwd": self.topdir + } + tasks_dict = { + "version": "2.0.0", + "tasks": [ + { + "label": install_label, + "type": "shell", + "command": run_do_install, + "problemMatcher": [] + }, + { + "label": deploy_label, + "type": "shell", + "command": "devtool", # TODO: Generate a self contained script which does what devtool target-deploy does but without connection to the bitbake server + "args": ["--bbpath", self.bbpath, "deploy-target", self.bpn, args.target, "--strip", "--no-host-check"], + "linux": { + "options": { + "env": bb_env + }}, + "options": bb_options, + "problemMatcher": [] + }, + { + "label": "install && deploy-target %s" % self.bpn, + "dependsOrder": "sequence", + "dependsOn": [ + install_label, + deploy_label + ], + "problemMatcher": [], + "group": { + "kind": "build", + "isDefault": True + } + } + ] + } + for pretty_id, start_script in gdb_cross.get_gdbserver_start_scripts(): + tasks_dict['tasks'].append( + { + "label": pretty_id, + "type": "shell", + "isBackground": True, + "command": start_script, + "problemMatcher": [ + { + "pattern": [ + { + "regexp": ".", + "file": 1, + "location": 2, + "message": 3 + } + ], + "background": { + "activeOnStart": True, + "beginsPattern": ".", + "endsPattern": ".", + } + } + ] + }) + tasks_file = 'tasks.json' + RecipeModified.update_json_file( + self.dot_code_dir, tasks_file, tasks_dict) + + def setup_ide(self, args, image, gdb_cross): + if self.build_tool == BuildTool.CMAKE: + self.__cmake_preset() + if self.build_tool == BuildTool.MESON: + self.__gen_meson_wrapper() + + if args.ide == 'code': + self.vscode_settings() + self.vscode_extensions() + self.vscode_c_cpp_properties() + if args.target: + self.vscode_launch(image, gdb_cross) + self.vscode_tasks(args, gdb_cross) + if args.ide == 'none' and self.build_tool == BuildTool.CMAKE: + self.none_launch(image, gdb_cross) + + +def ide_setup(args, config, basepath, workspace): + bootstap_tasks = [] + + tinfoil = setup_tinfoil(config_only=False, basepath=basepath) + try: + # Provide a rootfs and the corresponding debug symbols via rootfs-dbg or debuginfod + image_config = RecipeImage(args.image) + image_config.initialize(config, tinfoil) + bootstap_tasks += image_config.bootstrap_tasks + + target_device = TargetDevice(args) + + sdk_env = None + gdb_cross = None + if args.recipename in SHARED_SYSROOT_RECIPES: + ide_support = RecipeMetaIdeSupport() + ide_support.initialize(config, tinfoil) + bootstap_tasks += ide_support.bootstrap_tasks + + build_sysroots = RecipeBuildSysroots() + build_sysroots.initialize(config, tinfoil) + bootstap_tasks += build_sysroots.bootstrap_tasks + + sdk_env = SharedSysrootsEnv() + sdk_env.initialize(ide_support, build_sysroots) + else: + sdk_env = RecipeModified(args.recipename) + sdk_env.initialize(config, workspace, tinfoil) + bootstap_tasks += sdk_env.bootstrap_tasks + + gdb_multi_mode = not bool(args.ide == 'code') + gdb_cross = RecipeGdbCross( + args, sdk_env.target_arch, target_device, gdb_multi_mode) + gdb_cross.initialize(config, workspace, tinfoil) + bootstap_tasks += gdb_cross.bootstrap_tasks + finally: + tinfoil.shutdown() + + shared_sysroot = bool(args.recipename in SHARED_SYSROOT_RECIPES) + if not shared_sysroot: + sdk_env.debug_build_config(args) + + if not args.skip_bitbake: + bb_cmd = 'bitbake ' + if args.bitbake_k: + bb_cmd += "-k " + bb_cmd += ' '.join(bootstap_tasks) + exec_build_env_command(config.init_path, basepath, bb_cmd, watch=True) + + if shared_sysroot: + sdk_env.setup_ide(args) + else: + sdk_env.setup_ide(args, image_config, gdb_cross) + + +def get_default_ide(): + for an_ide in SUPPORTED_IDES[:-1]: + if shutil.which(an_ide): + return an_ide + return SUPPORTED_IDES[-1:] + + +def register_commands(subparsers, context): + """Register devtool subcommands from this plugin""" + parser_ide = subparsers.add_parser('ide', help='Setup the IDE (VSCode)', + description='Configure the IDE to work with the source code of a recipe.', + group='working', order=50, formatter_class=RawTextHelpFormatter) + parser_ide.add_argument( + 'recipename', help='Generate a IDE configuration in the workspace of the given recipe. ' + 'By default the workspace is configured to use the recipe sysroot prepared by devtool modify. ' + 'For some special recipes the generated configuration referes to the shared sysroots ' + 'provided by meta-ide-setup and build-sysroots recipes. ' + 'The following recipes use a shared sysroot: %s' % SHARED_SYSROOT_RECIPES) + parser_ide.add_argument( + 'image', help='The image running on the target device. This is required for remote debugging.' + 'It is important to deploy the image built by this command to the target device because ' + 'otherwise the debug symbols used on the build machine (rootfs-dbg) are probably out of sync ' + 'with the binaries executed on the target device.') + parser_ide.add_argument( + '-i', '--ide', choices=SUPPORTED_IDES, default=get_default_ide(), + help='Setup the configuration for this IDE') + parser_ide.add_argument( + '-t', '--target', default='root@192.168.7.2', + help='Live target machine running an ssh server: user@hostname.') + parser_ide.add_argument( + '-G', '--gdbserver-port-start', default="1234", help='port where gdbserver is listening.') + parser_ide.add_argument( + '-c', '--no-host-check', help='Disable ssh host key checking', action='store_true') + parser_ide.add_argument( + '-e', '--ssh-exec', help='Executable to use in place of ssh') + parser_ide.add_argument( + '-P', '--port', help='Specify ssh port to use for connection to the target') + parser_ide.add_argument( + '-I', '--key', help='Specify ssh private key for connection to the target') + parser_ide.add_argument( + '--skip-bitbake', help='Generate IDE configuration but skip calling bibtake to update the SDK.', action='store_true') + parser_ide.add_argument( + '-k', '--bitbake-k', help='Pass -k parameter to bitbake', action='store_true') + parser_ide.add_argument( + '--debug-build-config', help='Use debug build flags, for example set CMAKE_BUILD_TYPE=Debug', action='store_true') + parser_ide.set_defaults(func=ide_setup) + + # TODO: Better support for multiple recipes. E.g. a list of recipes with auto-detection for the image recipe or all modified recipes From patchwork Mon Aug 21 16:07:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8706EE49B3 for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) by mx.groups.io with SMTP id smtpd.web11.1049.1692634309056473550 for ; Mon, 21 Aug 2023 09:11:49 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=RzjwON2D; spf=pass (domain: gmail.com, ip: 209.85.208.46, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-52a0856b4fdso1676016a12.1 for ; Mon, 21 Aug 2023 09:11:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634307; x=1693239107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VaWuuIUtmaan9OQ7nZXD7CNTUSo25dmTfn2Y4hW5+Sg=; b=RzjwON2DSi9QPP+cFezeFYFedNnGqa1j3StpMlZVl2fVIfwXLVQBhX90R6UIcyGHhk 77234feIe+cq0kqZY5giIst5625bA3w6diYvsfENFAZ0XePFcTbpW52LTtFZQbGRovL9 Fr8wz8cy2ir4XGhIEru3xS5ZahiLzYvkhwdwOtG7n2tsSJsE+VQTRCYgreSimhfRTtFd Fm7XJxGC3vFEngeHxHBmK/jF8wPSJqPya5QBXJxF017RqcF98yUvyf/aQ0wb7/t/8/aQ tVZWWUASGnxTWSUubLL4pDn8w9xXHW/mZi97prke4M+2fv7Mj4twh4JeJw6jZrFD4Lwd 2yrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634307; x=1693239107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VaWuuIUtmaan9OQ7nZXD7CNTUSo25dmTfn2Y4hW5+Sg=; b=J7oulnsRFadC/XheIy70kkDufn1uJvuulk1XorkmuI515qkpYniIsSyxpeCDYZzyh6 q1o1WpkYAiB/NiFRqKzQO/0oaLVsnhjf0PMw6HvCOpvJYwOflWrp7sfWQHt3LV/YkCa9 9X39vCiHtI4EKRZrp2kLG0EajxZoOKZ16BOUqjox+E5d3i06LzeDgnzh6bSscA5sOEB6 1A1703rHjXZUck3m5BONIi1hOq0nRQPnRuopc9/UPXuLx+uK9jD54dazRVKUj1WD0Cxa sD2DX4JtoUlZHgHru9DfeKDSBHsSoFJnaIZFJGG5vtIae4XCVuB1EjW/yLzNSsZcce5b A/nQ== X-Gm-Message-State: AOJu0Yx9lPhKx29m4lhNwY0vjapyaEe5iN8SA3BTymz+e+r8P7wuVhdt nMPRCr40FORb9utaUOZWVTFxBijFbd4= X-Google-Smtp-Source: AGHT+IEKjoJgHqQjh5S3nlAyeAaXhLzYPYUEwhZlHAzHYEokI6YaZgkAl3JSKiWXhQjeocjJxmJQVw== X-Received: by 2002:a50:ee92:0:b0:525:469a:fc47 with SMTP id f18-20020a50ee92000000b00525469afc47mr4889215edr.22.1692634307114; Mon, 21 Aug 2023 09:11:47 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:46 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 07/14] tests: add a C++ example recipe Date: Mon, 21 Aug 2023 18:07:10 +0200 Message-ID: <20230821161120.171956-8-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186469 This simple C++ project supports compilation with cmake and with meson. It's supposed to be used with oe-selftest for the devtool ide plugin. Signed-off-by: Adrian Freihofer --- meta-selftest/recipes-test/cpp/.gitignore | 1 + .../recipes-test/cpp/cmake-example.bb | 17 ++++++ .../recipes-test/cpp/cmake-example/run-ptest | 10 ++++ .../recipes-test/cpp/cpp-example.inc | 24 ++++++++ .../recipes-test/cpp/files/CMakeLists.txt | 60 +++++++++++++++++++ .../cpp/files/cpp-example-lib.cpp | 17 ++++++ .../cpp/files/cpp-example-lib.hpp | 16 +++++ .../recipes-test/cpp/files/cpp-example.cpp | 16 +++++ .../recipes-test/cpp/files/meson.build | 34 +++++++++++ .../cpp/files/test-cpp-example.cpp | 19 ++++++ .../recipes-test/cpp/meson-example.bb | 17 ++++++ .../recipes-test/cpp/meson-example/run-ptest | 10 ++++ 12 files changed, 241 insertions(+) create mode 100644 meta-selftest/recipes-test/cpp/.gitignore create mode 100644 meta-selftest/recipes-test/cpp/cmake-example.bb create mode 100644 meta-selftest/recipes-test/cpp/cmake-example/run-ptest create mode 100644 meta-selftest/recipes-test/cpp/cpp-example.inc create mode 100644 meta-selftest/recipes-test/cpp/files/CMakeLists.txt create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example.cpp create mode 100644 meta-selftest/recipes-test/cpp/files/meson.build create mode 100644 meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp create mode 100644 meta-selftest/recipes-test/cpp/meson-example.bb create mode 100644 meta-selftest/recipes-test/cpp/meson-example/run-ptest diff --git a/meta-selftest/recipes-test/cpp/.gitignore b/meta-selftest/recipes-test/cpp/.gitignore new file mode 100644 index 00000000000..30d388a12b7 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/.gitignore @@ -0,0 +1 @@ +build* \ No newline at end of file diff --git a/meta-selftest/recipes-test/cpp/cmake-example.bb b/meta-selftest/recipes-test/cpp/cmake-example.bb new file mode 100644 index 00000000000..96d543180b4 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/cmake-example.bb @@ -0,0 +1,17 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +SUMMARY = "A C++ example compiled with cmake." + +inherit cmake + +require cpp-example.inc + +SRC_URI += "\ + file://CMakeLists.txt \ +" + +FILES:${PN}-ptest += "${bindir}/test-cmake-example" diff --git a/meta-selftest/recipes-test/cpp/cmake-example/run-ptest b/meta-selftest/recipes-test/cpp/cmake-example/run-ptest new file mode 100644 index 00000000000..94b620a1984 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/cmake-example/run-ptest @@ -0,0 +1,10 @@ +#!/bin/sh +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +test-cmake-example + +# Note: run-ptests exits with exit value from test-cmake-example diff --git a/meta-selftest/recipes-test/cpp/cpp-example.inc b/meta-selftest/recipes-test/cpp/cpp-example.inc new file mode 100644 index 00000000000..39c61cf4ceb --- /dev/null +++ b/meta-selftest/recipes-test/cpp/cpp-example.inc @@ -0,0 +1,24 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" + +inherit ptest + +DEPENDS += "json-c" + +PV = "1.0" + +S = "${WORKDIR}" + +SRC_URI = "\ + file://cpp-example.cpp \ + file://cpp-example-lib.hpp \ + file://cpp-example-lib.cpp \ + file://test-cpp-example.cpp \ + file://run-ptest \ +" diff --git a/meta-selftest/recipes-test/cpp/files/CMakeLists.txt b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt new file mode 100644 index 00000000000..839aa59b5e3 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt @@ -0,0 +1,60 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +cmake_minimum_required(VERSION 3.22) + +project(cmake-example + VERSION 1.0.0 + LANGUAGES CXX +) + +option(BUILD_SHARED_LIBS "Build using shared libraries" ON) + +set(CMAKE_CXX_STANDARD 17) +set(CMAKE_CXX_STANDARD_REQUIRED On) +set(CMAKE_CXX_EXTENSIONS Off) + +include(GNUInstallDirs) + +# Find json-c +# find_package(PkgConfig REQUIRED) +# pkg_check_modules(JSONC REQUIRED json-c) +find_package(json-c) + +# A simple library linking json-c library found by pkgconfig +add_library(cmake-example-lib cpp-example-lib.cpp cpp-example-lib.hpp) +set_target_properties(cmake-example-lib PROPERTIES + VERSION ${PROJECT_VERSION} + SOVERSION ${PROJECT_VERSION_MAJOR} +) +target_link_libraries(cmake-example-lib PRIVATE json-c::json-c) +# target_link_libraries(cmake-example-lib ${JSONC_LIBRARIES}) +# target_include_directories(cmake-example-lib PUBLIC ${JSONC_INCLUDE_DIRS}) +# target_compile_options(cmake-example-lib PUBLIC ${JSONC_CFLAGS_OTHER}) +install(TARGETS cmake-example-lib + INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} + ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} + LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} +) + +# A simple executable linking the library +add_executable(cmake-example cpp-example.cpp) +target_link_libraries(cmake-example PRIVATE cmake-example-lib) + +install(TARGETS cmake-example + RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} +) + +# A simple test executable for testing the library +add_executable(test-cmake-example test-cpp-example.cpp) +target_link_libraries(test-cmake-example PRIVATE cmake-example-lib) + +install(TARGETS test-cmake-example + RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} +) + +include(CTest) +add_test(NAME test-cmake-example COMMAND test-cmake-example) diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp new file mode 100644 index 00000000000..29fd10fb7e1 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp @@ -0,0 +1,17 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#include +#include +#include "cpp-example-lib.hpp" + +const std::string& CppExample::get_string() { + return test_string; +} + +const char* CppExample::get_json_c_version() { + return json_c_version(); +} diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp new file mode 100644 index 00000000000..8a87e6b510a --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp @@ -0,0 +1,16 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#pragma once + +#include + +struct CppExample { + inline static const std::string test_string = "cpp-example-lib Magic: 123456789"; + + const std::string& get_string(); + const char* get_json_c_version(); +}; diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example.cpp b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp new file mode 100644 index 00000000000..de7f2106dec --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp @@ -0,0 +1,16 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#include "cpp-example-lib.hpp" + +#include + +int main() { + auto cpp_example = CppExample(); + std::cout << "C++ example linking " << cpp_example.get_string() << std::endl; + std::cout << "Linking json-c version " << cpp_example.get_json_c_version() << std::endl; + return 0; +} diff --git a/meta-selftest/recipes-test/cpp/files/meson.build b/meta-selftest/recipes-test/cpp/files/meson.build new file mode 100644 index 00000000000..7753da69721 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/meson.build @@ -0,0 +1,34 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +project('meson-example', 'cpp', + version: '1.0.0', + default_options: ['cpp_std=c++17'] + ) + + +jsoncdep = dependency('json-c') + +mesonexlib = shared_library('mesonexlib', + 'cpp-example-lib.cpp', 'cpp-example-lib.hpp', + version: meson.project_version(), + soversion: meson.project_version().split('.')[0], + dependencies : jsoncdep, + install : true + ) + +executable('mesonex', + 'cpp-example.cpp', + link_with : mesonexlib, + install : true + ) + +test_mesonex = executable('test-mesonex', + 'test-cpp-example.cpp', + link_with : mesonexlib, + install : true +) +test('meson example test', test_mesonex) diff --git a/meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp b/meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp new file mode 100644 index 00000000000..ca72e11a9c4 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp @@ -0,0 +1,19 @@ +/* +* Copyright OpenEmbedded Contributors +* +* SPDX-License-Identifier: MIT +*/ + +#include "cpp-example-lib.hpp" + +#include + +int main() { + auto cpp_example = CppExample(); + auto ret_string = cpp_example.get_string(); + if(0 == ret_string.compare(CppExample::test_string)) { + std::cout << "PASS: " << ret_string << " = " << CppExample::test_string << std::endl; + } else { + std::cout << "FAIL: " << ret_string << " != " << CppExample::test_string << std::endl; + } +} diff --git a/meta-selftest/recipes-test/cpp/meson-example.bb b/meta-selftest/recipes-test/cpp/meson-example.bb new file mode 100644 index 00000000000..f15dbb50731 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/meson-example.bb @@ -0,0 +1,17 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +SUMMARY = "A C++ example compiled with meson." + +inherit pkgconfig meson + +require cpp-example.inc + +SRC_URI += "\ + file://meson.build \ +" + +FILES:${PN}-ptest += "${bindir}/test-mesonex" diff --git a/meta-selftest/recipes-test/cpp/meson-example/run-ptest b/meta-selftest/recipes-test/cpp/meson-example/run-ptest new file mode 100644 index 00000000000..b1804f00961 --- /dev/null +++ b/meta-selftest/recipes-test/cpp/meson-example/run-ptest @@ -0,0 +1,10 @@ +#!/bin/sh +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +test-mesonex + +# Note: run-ptests exits with exit value from test-mesonex From patchwork Mon Aug 21 16:07:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29240 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94701EE49B0 for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by mx.groups.io with SMTP id smtpd.web11.1050.1692634309684250499 for ; Mon, 21 Aug 2023 09:11:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=CceFWsXN; spf=pass (domain: gmail.com, ip: 209.85.208.43, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-52683b68c2fso4314457a12.0 for ; Mon, 21 Aug 2023 09:11:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634308; x=1693239108; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ErVR+4mr/bEwryE3Z1g4lUfVjCMEfJBdh+oxF159VxM=; b=CceFWsXNFgp/eRTnHqUBfkyK/zJ9bVf8QMUHEqStXvw/uT7OwPQ5QZK6neC1X7Vp2m gukrqTECq99QeWiL4XBwHkYT3ynPvxGRIZf8FhEZ/jzs49TfH/l4sCrgB1glr/Mc9you FpPVxc1sKkHak78+y4M0sqs7RT4djVZYLlcBeUy/v/g5HHOYDnBYfcgp98bu5Nm5kbxg XeNxXSboLPLqJ8ZscwxDCxBEEQfF1Gu0uwq0bgpPn+MpHzbqdc6Q2G/qLv55uDTMcK4x O0W3HAsTnXWaD7jT7vMgOFoyo3Dke0tJh3LMdy7HyFfMRxfA+A5ZnhNZhh6eecLYtxQC 9uYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634308; x=1693239108; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ErVR+4mr/bEwryE3Z1g4lUfVjCMEfJBdh+oxF159VxM=; b=kz6jokt7oPOp8FAfYkZ5jxCjeO3JJg9jtWBgHUV1RzVK2LgLIhlVYhGRV2jR8TE6Zy yoV/FQWX31Y8893I6Dmk8I56YYMLn3rap6KkkVIO38hw29XfPUJKh1UKir060fEWIXPD t5xGUc+232JL3cCnlqDB1T6p+eRJ4FBfozztYL5+iRjKKlSTmGyJuettNSaWrAau1Fz/ QdM3aKtWLnFDnd/bLSWVl5c/QzDMHAi5BlNvs1lx7lEBY6mWZBxOs7FJIKpAr+y6A3dG v+Csq4RZof8/XZUY9GTeDXegS3rZwuUNZcFcQFdcuIW+4BHQqpezsiiglK0A6kptMv4a GJQw== X-Gm-Message-State: AOJu0Yx0kIP0iz3zkNB0jiFdBeP9j5IIaeGiFrJO9U78nPkfotDEBX9J Fyfu+dNqsdr/6U7vFIBh2VVMTgbmTxM= X-Google-Smtp-Source: AGHT+IFA1YmBjzo0XPvj0wPmMIFNiAAdGNjNetK0stuvP6w1uTvmLeHFvTBn0us4cOpdmxlTMsXdnw== X-Received: by 2002:a05:6402:1610:b0:525:5886:2f69 with SMTP id f16-20020a056402161000b0052558862f69mr5661396edv.36.1692634307753; Mon, 21 Aug 2023 09:11:47 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:47 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 08/14] cmake-example: workaround for pseudo breakeage Date: Mon, 21 Aug 2023 18:07:11 +0200 Message-ID: <20230821161120.171956-9-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186471 --- meta-selftest/recipes-test/cpp/cmake-example.bb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta-selftest/recipes-test/cpp/cmake-example.bb b/meta-selftest/recipes-test/cpp/cmake-example.bb index 96d543180b4..fbf1f266721 100644 --- a/meta-selftest/recipes-test/cpp/cmake-example.bb +++ b/meta-selftest/recipes-test/cpp/cmake-example.bb @@ -15,3 +15,9 @@ SRC_URI += "\ " FILES:${PN}-ptest += "${bindir}/test-cmake-example" + +# This is a workaround for packages with sources in-tree +# Without this bitbake ... does not work after running the install script. +# newenv['PSEUDO_IGNORE_PATHS'] = newenv['PSEUDO_IGNORE_PATHS'] + "" +# path mismatch [3 links]: ino 37529096 db '/home/adrian/projects/oss/meta-yocto-upstream/projects/poky-oe-glibc-sd/tmp/work/cortexa57-poky-linux/cmake-example/1.0-r0/package/usr/src/debug/cmake-example/1.0-r0/oe-local-files/cmake-example-lib.cpp' req '/home/adrian/projects/oss/meta-yocto-upstream/projects/poky-oe-glibc-sd/workspace/sources/cmake-example/oe-local-files/cmake-example-lib.cpp'. +PACKAGE_DEBUG_SPLIT_STYLE = "debug-without-src" From patchwork Mon Aug 21 16:07:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29242 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9508CEE49B2 for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by mx.groups.io with SMTP id smtpd.web11.1051.1692634310533466323 for ; Mon, 21 Aug 2023 09:11:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=a3xEK5NS; spf=pass (domain: gmail.com, ip: 209.85.208.44, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-529fb2c6583so2460056a12.1 for ; Mon, 21 Aug 2023 09:11:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634309; x=1693239109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=19fLgghekqWM22ZBRBUMVm4pfisluZVtZiAq51rVSl8=; b=a3xEK5NSs7DCDY/WzVIag5ca5SGAQD9OxnpOr6fEKIB2l8daRH1Z4pa7PQq4Wsw22a FCudXjM0b7IH0qgC19slZ74ndAuXarWmah7Kpai197i1Q5b0p/ZC8xEaqq/GzhM6uZJi Otq8xTUdvnhOCVCNse9G5sDdU+NWl8ehG5+BcIOT5JY1z9vymM4ujlhNpVLo5gBliBUM zkGMw4Tq0OO2rtaUEixcUH/ISTcaojnppdeOMn2p5pfmqgU/vomAsr2XqRp+hs6sh+DI dMnKmtCtMnXhRe+p9sHbeYQLOpQEZG6hfiOJTJYzE/Adfojwa9SJyC50pOviJEJINT1S Zugg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634309; x=1693239109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=19fLgghekqWM22ZBRBUMVm4pfisluZVtZiAq51rVSl8=; b=B3buS6iGquXCx91WJ1X/7KCiFYpNHxsSjZXMVCbEmXX1jxMoYra8p3Cqt/KxN09c6W gVfUzenFV6cbPJANGceOv4hna9HBBeNthnbm/okmVY/Zqu0W+9vhe/btpkJ6RTASTX7+ KKsRfY9CAecf/xk5QvJILkA5U4LVL+/HrLZp3Jc1UT9pMqqVxfhES/L9hYbymqGGvfqp cWdjXeHrlF385hEgUfhrl+JgeVw4YEc5DfbDQAG05gCawjsdsw7linPwJ4JRRQapv0mr 0Ch0QCBz5T6qdzHPKwnAjrtgTGCi8ifdFSRPURR1JHMQTrm0h/j3hecV0aXYdeKrpQKm uokA== X-Gm-Message-State: AOJu0Yxdad6/mBXTI9RNfx9+FOWR98AEsA5wbGjUZiFXLn3dLVN5jkfO Y0RGk+KIlHmIz+Z7/CNoyPxXkMsIwrU= X-Google-Smtp-Source: AGHT+IHSD/rNljqR9OocZ6M45z9LqxvaJZEpU2QxKPU7g2bf5NPqzAbslzftUSLhHs1kj5ZRuyw5Ew== X-Received: by 2002:a05:6402:164e:b0:522:cef7:83c3 with SMTP id s14-20020a056402164e00b00522cef783c3mr5327638edx.8.1692634308710; Mon, 21 Aug 2023 09:11:48 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:48 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 09/14] refactor: make multiprocess_launch callable without d Date: Mon, 21 Aug 2023 18:07:12 +0200 Message-ID: <20230821161120.171956-10-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186472 Signed-off-by: Adrian Freihofer --- meta/lib/oe/package.py | 5 +++-- meta/lib/oe/utils.py | 12 +++++++++--- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 9d70925b9b7..ffca7559ba7 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -164,7 +164,8 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp # ...but is it ELF, and is it already stripped? checkelf.append(file) inodecache[file] = s.st_ino - results = oe.utils.multiprocess_launch(is_elf, checkelf, d) + max_process = int(d.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) + results = oe.utils.multiprocess_launch_mp(is_elf, checkelf, max_process) for (file, elf_file) in results: #elf_file = is_elf(file) if elf_file & 1: @@ -192,7 +193,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp elf_file = int(elffiles[file]) sfiles.append((file, elf_file, strip_cmd)) - oe.utils.multiprocess_launch(runstrip, sfiles, d) + oe.utils.multiprocess_launch_mp(runstrip, sfiles, max_process) def file_translate(file): diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py index 69ca8987f3d..430e874d90a 100644 --- a/meta/lib/oe/utils.py +++ b/meta/lib/oe/utils.py @@ -264,10 +264,17 @@ def execute_pre_post_process(d, cmds): bb.note("Executing %s ..." % cmd) bb.build.exec_func(cmd, d) -# For each item in items, call the function 'target' with item as the first +def multiprocess_launch_max_process(d): + return int(d.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) + +def multiprocess_launch(target, items, d, extraargs=None): + max_process = multiprocess_launch_max_process(d) + return multiprocess_launch_mp(target, items, max_process, extraargs) + +# For each item in items, call the function 'target' with item as the first # argument, extraargs as the other arguments and handle any exceptions in the # parent thread -def multiprocess_launch(target, items, d, extraargs=None): +def multiprocess_launch_mp(target, items, max_process, extraargs=None): class ProcessLaunch(multiprocessing.Process): def __init__(self, *args, **kwargs): @@ -302,7 +309,6 @@ def multiprocess_launch(target, items, d, extraargs=None): self.update() return self._result - max_process = int(d.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) launched = [] errors = [] results = [] From patchwork Mon Aug 21 16:07:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86EC2EE49AF for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) by mx.groups.io with SMTP id smtpd.web10.1061.1692634311452692323 for ; Mon, 21 Aug 2023 09:11:51 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=HY5o3ajD; spf=pass (domain: gmail.com, ip: 209.85.208.46, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-52683da3f5cso4290694a12.3 for ; Mon, 21 Aug 2023 09:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634309; x=1693239109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oh/oWmI5NxCsZhxz9NsVtSOvMlWbflWmbtdhRAaNyO8=; b=HY5o3ajDF65K5uQnzVh7Vio6C7Oe1CTiKH65ULmLqQ981VCt0IEqqFx4qnP1j/DDTU HJzMJCtNIJnxdfli6yoCSzh9XWMu85bOFM/2Sv8P+bmLf9Hc0EDTjk4DFxJRdorI/5jr cNJdVkS0KLdHk9bE8LqzZV3l3HJDMRtdxKfxDrOiQ7rmGC/KMn6Cx4qZAYYNNZojv5A4 qmWdw5nWCZpIuq23XVflPnTXNMwc+muFQ3Jz3+122ef69RUSPqR3QP+IT3NHVVNsIe1x yngAxOL8CzgZpBPb9NplhEVk/716tiYcQqfJEk+SB9mRMokhD3mCCYusY47mWd5Vs3d0 mpeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634309; x=1693239109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oh/oWmI5NxCsZhxz9NsVtSOvMlWbflWmbtdhRAaNyO8=; b=PAs4a46Kx0arTrAZWJF1guT0shnM3aQeygFYRgsyON7qP8R1Q6Dj0zINSF2FWaX3zx ndO0sIQ9yByIRGMtgeu5FGy7mfQF3B4vMsUKqAgu/wPhFVYJ6My1sqZV1kCXyHCNnu6V PXB5BwlI1XYfFENDImUmOZi0zmu1Tchtnao7Xa/QR67qqecgP9wY52s1olb+Uqh4fEea B7l2V7xltQMxIPUZElAEdKgAqQg0nE/ATbTc1ZP+evcOXE9ugnO1WynVtG3Dk5J1ApH9 VWHdqV8WrpVTXYmCsHQrpPYTVdf0MGC7Zs/pMU5RZuBAubE0M1WQNPEKHiU1GYP+quyA Po8w== X-Gm-Message-State: AOJu0Yw/CPldyB3qgPfmRN7btWv+DL2G8Lds1RBdrUkVJRCk/6WgFYT1 D89stt1ua7LL9pu5UJzvQl0YpNqG/L0= X-Google-Smtp-Source: AGHT+IHuwuXspMMq/m4MW3QrlpVKXXR1rRP+dueurwy4HT0T/fiUm4VjOU46FT4dt5V4XxY/iU/puA== X-Received: by 2002:aa7:c6c1:0:b0:525:7234:52bb with SMTP id b1-20020aa7c6c1000000b00525723452bbmr5137381eds.24.1692634309482; Mon, 21 Aug 2023 09:11:49 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:49 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 10/14] refactor: make strip_execs callable without d Date: Mon, 21 Aug 2023 18:07:13 +0200 Message-ID: <20230821161120.171956-11-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186473 Signed-off-by: Adrian Freihofer --- meta/classes-global/staging.bbclass | 3 ++- meta/lib/oe/package.py | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/meta/classes-global/staging.bbclass b/meta/classes-global/staging.bbclass index 3a300c32e7c..7f4075a5f6e 100644 --- a/meta/classes-global/staging.bbclass +++ b/meta/classes-global/staging.bbclass @@ -92,7 +92,8 @@ python sysroot_strip () { qa_already_stripped = 'already-stripped' in (d.getVar('INSANE_SKIP:' + pn) or "").split() strip_cmd = d.getVar("STRIP") - oe.package.strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, + max_process = oe.utils.multiprocess_launch_max_process(d) + oe.package.strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, max_process, qa_already_stripped=qa_already_stripped) } diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index ffca7559ba7..1dd20f85ebd 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -114,7 +114,7 @@ def is_static_lib(path): return start == magic return False -def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripped=False): +def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, max_process, qa_already_stripped=False): """ Strip executable code (like executables, shared libraries) _in_place_ - Based on sysroot_strip in staging.bbclass @@ -122,6 +122,7 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp :param strip_cmd: Strip command (usually ${STRIP}) :param libdir: ${libdir} - strip .so files in this directory :param base_libdir: ${base_libdir} - strip .so files in this directory + :param max_process: number of stripping processes started in parallel :param qa_already_stripped: Set to True if already-stripped' in ${INSANE_SKIP} This is for proper logging and messages only. """ @@ -164,7 +165,6 @@ def strip_execs(pn, dstdir, strip_cmd, libdir, base_libdir, d, qa_already_stripp # ...but is it ELF, and is it already stripped? checkelf.append(file) inodecache[file] = s.st_ino - max_process = int(d.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) results = oe.utils.multiprocess_launch_mp(is_elf, checkelf, max_process) for (file, elf_file) in results: #elf_file = is_elf(file) From patchwork Mon Aug 21 16:07:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 888D0EE49AE for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by mx.groups.io with SMTP id smtpd.web10.1063.1692634312152190309 for ; Mon, 21 Aug 2023 09:11:52 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=KBy7gmRX; spf=pass (domain: gmail.com, ip: 209.85.208.49, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-52889bc61b6so4512788a12.0 for ; Mon, 21 Aug 2023 09:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634310; x=1693239110; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jy8i1/A44FHlgQLN08tx4d2NioWfPZimdixvZe0CbjI=; b=KBy7gmRXJm7jlqAzEzZvf+P3Rq+ShUEkyaFcByivvSd7tdD9OPt+uN1OvO4BPLPOs5 km2wqhVR04tKoVArmq+80+IE0XM2oDzBkXKHFz9xF0tOid0/nKZLPh3BnSOjXngLPPzQ ZwdlDeey+LCCIA9Zv1nDiP/e/7XPeGWHmQpE6EKbajZvErq8bYKaNPcNrv2c8iXKIiMU TeZQLWCjejCKSa2GxEucXsm1EFI0b5+40A4dkwkb7i6cOQHbYT04X/ygSSm6GuBCJwpN szvoc0XaKcqPf6LAZWmlbLeRbo+fSkGBi6I2mJImCH2e9jLKPFFqSupORLsJjrAt4PMP lGpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634310; x=1693239110; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jy8i1/A44FHlgQLN08tx4d2NioWfPZimdixvZe0CbjI=; b=Tt9j+grT/6HVGFywug4ukRtj+DR8xivIbTxrBBMh0rM8zfAwvNtLIs1ClHT3GMZhxv U0yRab6ia8fjO4Zy81lOzw2+cmbNOWeS/2LqNZDBYgMu3fB5m0bPpGUS4byZEXkTbsGe ZdUcntE0ta2euEj12EoYZLedOCNSwi6uY5p+GWcv9nIqKvsLH+WALdGW2GY7aG+4Z58d ZvXO0pUfv1qHkJNZ8YUeCwmAm98REROwpmmFY8WOEYsGex2kKF1bzWUa5ZeQmKRrmPWS cxx5onV7Wnc/j+tF+iodutFHQkYpptfsCiTpDC41mD4OWGveugtj1uxNrhJFlcld3ggm Fdqg== X-Gm-Message-State: AOJu0Yw1ZquBz7R4c9N/H6aVq0hlIBIojU/UGFHWnjrtgjVIrqUm/CtG S5mDVLBG6M4V3zx//0QzLzruX2MSwOk= X-Google-Smtp-Source: AGHT+IGqpacslhuovCjqibsHN/CSNdzT2fUaeFEyuIFxK58up3Yvz4A1GgrfP5cZI18DK4BhtAoWHQ== X-Received: by 2002:a05:6402:12cb:b0:528:89d0:90b7 with SMTP id k11-20020a05640212cb00b0052889d090b7mr6091572edx.16.1692634310204; Mon, 21 Aug 2023 09:11:50 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:49 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 11/14] devtool: refactor deploy-target Date: Mon, 21 Aug 2023 18:07:14 +0200 Message-ID: <20230821161120.171956-12-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186474 Signed-off-by: Adrian Freihofer --- scripts/lib/devtool/__init__.py | 5 +- scripts/lib/devtool/deploy.py | 230 +++++++++++++++++--------------- 2 files changed, 124 insertions(+), 111 deletions(-) diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py index 702db669de3..7d64547616e 100644 --- a/scripts/lib/devtool/__init__.py +++ b/scripts/lib/devtool/__init__.py @@ -78,12 +78,15 @@ def exec_fakeroot(d, cmd, **kwargs): """Run a command under fakeroot (pseudo, in fact) so that it picks up the appropriate file permissions""" # Grab the command and check it actually exists fakerootcmd = d.getVar('FAKEROOTCMD') + fakerootenv = d.getVar('FAKEROOTENV') + exec_fakeroot_no_d(fakerootcmd, fakerootenv, cmd, kwargs) + +def exec_fakeroot_no_d(fakerootcmd, fakerootenv, cmd, **kwargs): if not os.path.exists(fakerootcmd): logger.error('pseudo executable %s could not be found - have you run a build yet? pseudo-native should install this and if you have run any build then that should have been built') return 2 # Set up the appropriate environment newenv = dict(os.environ) - fakerootenv = d.getVar('FAKEROOTENV') for varvalue in fakerootenv.split(): if '=' in varvalue: splitval = varvalue.split('=', 1) diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py index e14a5874177..ea7e2cb1ae8 100644 --- a/scripts/lib/devtool/deploy.py +++ b/scripts/lib/devtool/deploy.py @@ -16,7 +16,7 @@ import bb.utils import argparse_oe import oe.types -from devtool import exec_fakeroot, setup_tinfoil, check_workspace_recipe, DevtoolError +from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, DevtoolError logger = logging.getLogger('devtool') @@ -133,16 +133,11 @@ def _prepare_remote_script(deploy, verbose=False, dryrun=False, undeployall=Fals return '\n'.join(lines) - - -def deploy(args, config, basepath, workspace): - """Entry point for the devtool 'deploy' subcommand""" +def deploy_cached(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args): import math import oe.recipeutils import oe.package - check_workspace_recipe(workspace, args.recipename, checksrc=False) - try: host, destdir = args.target.split(':') except ValueError: @@ -152,116 +147,131 @@ def deploy(args, config, basepath, workspace): if not destdir.endswith('/'): destdir += '/' + recipe_outdir = srcdir + if not os.path.exists(recipe_outdir) or not os.listdir(recipe_outdir): + raise DevtoolError('No files to deploy - have you built the %s ' + 'recipe? If so, the install step has not installed ' + 'any files.' % args.recipename) + + if args.strip and not args.dry_run: + # Fakeroot copy to new destination + srcdir = recipe_outdir + recipe_outdir = os.path.join(workdir, 'devtool-deploy-target-stripped') + if os.path.isdir(recipe_outdir): + exec_fakeroot_no_d(fakerootcmd, fakerootenv, "rm -rf %s" % recipe_outdir, shell=True) + exec_fakeroot_no_d(fakerootcmd, fakerootenv, "cp -af %s %s" % (os.path.join(srcdir, '.'), recipe_outdir), shell=True) + os.environ['PATH'] = ':'.join([os.environ['PATH'], path or '']) + oe.package.strip_execs(args.recipename, recipe_outdir, strip_cmd, libdir, base_libdir, max_process) + + filelist = [] + inodes = set({}) + ftotalsize = 0 + for root, _, files in os.walk(recipe_outdir): + for fn in files: + fstat = os.lstat(os.path.join(root, fn)) + # Get the size in kiB (since we'll be comparing it to the output of du -k) + # MUST use lstat() here not stat() or getfilesize() since we don't want to + # dereference symlinks + if fstat.st_ino in inodes: + fsize = 0 + else: + fsize = int(math.ceil(float(fstat.st_size)/1024)) + inodes.add(fstat.st_ino) + ftotalsize += fsize + # The path as it would appear on the target + fpath = os.path.join(destdir, os.path.relpath(root, recipe_outdir), fn) + filelist.append((fpath, fsize)) + + if args.dry_run: + print('Files to be deployed for %s on target %s:' % (args.recipename, args.target)) + for item, _ in filelist: + print(' %s' % item) + return 0 + + extraoptions = '' + if args.no_host_check: + extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' + if not args.show_status: + extraoptions += ' -q' + + scp_sshexec = '' + ssh_sshexec = 'ssh' + if args.ssh_exec: + scp_sshexec = "-S %s" % args.ssh_exec + ssh_sshexec = args.ssh_exec + scp_port = '' + ssh_port = '' + if args.port: + scp_port = "-P %s" % args.port + ssh_port = "-p %s" % args.port + + if args.key: + extraoptions += ' -i %s' % args.key + + # In order to delete previously deployed files and have the manifest file on + # the target, we write out a shell script and then copy it to the target + # so we can then run it (piping tar output to it). + # (We cannot use scp here, because it doesn't preserve symlinks.) + tmpdir = tempfile.mkdtemp(prefix='devtool') + try: + tmpscript = '/tmp/devtool_deploy.sh' + tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list') + shellscript = _prepare_remote_script(deploy=True, + verbose=args.show_status, + nopreserve=args.no_preserve, + nocheckspace=args.no_check_space) + # Write out the script to a file + with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f: + f.write(shellscript) + # Write out the file list + with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f: + f.write('%d\n' % ftotalsize) + for fpath, fsize in filelist: + f.write('%s %d\n' % (fpath, fsize)) + # Copy them to the target + ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True) + if ret != 0: + raise DevtoolError('Failed to copy script to %s - rerun with -s to ' + 'get a complete error message' % args.target) + finally: + shutil.rmtree(tmpdir) + + # Now run the script + ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, 'tar cf - . | %s %s %s %s \'sh %s %s %s %s\'' % (ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) + if ret != 0: + raise DevtoolError('Deploy failed - rerun with -s to get a complete ' + 'error message') + + logger.info('Successfully deployed %s' % recipe_outdir) + + files_list = [] + for root, _, files in os.walk(recipe_outdir): + for filename in files: + filename = os.path.relpath(os.path.join(root, filename), recipe_outdir) + files_list.append(os.path.join(destdir, filename)) + +def deploy(args, config, basepath, workspace): + """Entry point for the devtool 'deploy' subcommand""" + check_workspace_recipe(workspace, args.recipename, checksrc=False) + tinfoil = setup_tinfoil(basepath=basepath) try: try: rd = tinfoil.parse_recipe(args.recipename) + + srcdir = rd.getVar('D') + workdir = rd.getVar('WORKDIR') + path = rd.getVar('PATH') + strip_cmd = rd.getVar('STRIP') + libdir = rd.getVar('libdir'), + base_libdir = rd.getVar('base_libdir') + max_process = int(rd.getVar("BB_NUMBER_THREADS") or os.cpu_count() or 1) + fakerootcmd = rd.getVar('FAKEROOTCMD') + fakerootenv = rd.getVar('FAKEROOTENV') + deploy_cached(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args) except Exception as e: raise DevtoolError('Exception parsing recipe %s: %s' % (args.recipename, e)) - recipe_outdir = rd.getVar('D') - if not os.path.exists(recipe_outdir) or not os.listdir(recipe_outdir): - raise DevtoolError('No files to deploy - have you built the %s ' - 'recipe? If so, the install step has not installed ' - 'any files.' % args.recipename) - - if args.strip and not args.dry_run: - # Fakeroot copy to new destination - srcdir = recipe_outdir - recipe_outdir = os.path.join(rd.getVar('WORKDIR'), 'devtool-deploy-target-stripped') - if os.path.isdir(recipe_outdir): - exec_fakeroot(rd, "rm -rf %s" % recipe_outdir, shell=True) - exec_fakeroot(rd, "cp -af %s %s" % (os.path.join(srcdir, '.'), recipe_outdir), shell=True) - os.environ['PATH'] = ':'.join([os.environ['PATH'], rd.getVar('PATH') or '']) - oe.package.strip_execs(args.recipename, recipe_outdir, rd.getVar('STRIP'), rd.getVar('libdir'), - rd.getVar('base_libdir'), rd) - - filelist = [] - inodes = set({}) - ftotalsize = 0 - for root, _, files in os.walk(recipe_outdir): - for fn in files: - fstat = os.lstat(os.path.join(root, fn)) - # Get the size in kiB (since we'll be comparing it to the output of du -k) - # MUST use lstat() here not stat() or getfilesize() since we don't want to - # dereference symlinks - if fstat.st_ino in inodes: - fsize = 0 - else: - fsize = int(math.ceil(float(fstat.st_size)/1024)) - inodes.add(fstat.st_ino) - ftotalsize += fsize - # The path as it would appear on the target - fpath = os.path.join(destdir, os.path.relpath(root, recipe_outdir), fn) - filelist.append((fpath, fsize)) - - if args.dry_run: - print('Files to be deployed for %s on target %s:' % (args.recipename, args.target)) - for item, _ in filelist: - print(' %s' % item) - return 0 - - extraoptions = '' - if args.no_host_check: - extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' - if not args.show_status: - extraoptions += ' -q' - - scp_sshexec = '' - ssh_sshexec = 'ssh' - if args.ssh_exec: - scp_sshexec = "-S %s" % args.ssh_exec - ssh_sshexec = args.ssh_exec - scp_port = '' - ssh_port = '' - if args.port: - scp_port = "-P %s" % args.port - ssh_port = "-p %s" % args.port - - if args.key: - extraoptions += ' -i %s' % args.key - - # In order to delete previously deployed files and have the manifest file on - # the target, we write out a shell script and then copy it to the target - # so we can then run it (piping tar output to it). - # (We cannot use scp here, because it doesn't preserve symlinks.) - tmpdir = tempfile.mkdtemp(prefix='devtool') - try: - tmpscript = '/tmp/devtool_deploy.sh' - tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list') - shellscript = _prepare_remote_script(deploy=True, - verbose=args.show_status, - nopreserve=args.no_preserve, - nocheckspace=args.no_check_space) - # Write out the script to a file - with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f: - f.write(shellscript) - # Write out the file list - with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f: - f.write('%d\n' % ftotalsize) - for fpath, fsize in filelist: - f.write('%s %d\n' % (fpath, fsize)) - # Copy them to the target - ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True) - if ret != 0: - raise DevtoolError('Failed to copy script to %s - rerun with -s to ' - 'get a complete error message' % args.target) - finally: - shutil.rmtree(tmpdir) - - # Now run the script - ret = exec_fakeroot(rd, 'tar cf - . | %s %s %s %s \'sh %s %s %s %s\'' % (ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True) - if ret != 0: - raise DevtoolError('Deploy failed - rerun with -s to get a complete ' - 'error message') - - logger.info('Successfully deployed %s' % recipe_outdir) - - files_list = [] - for root, _, files in os.walk(recipe_outdir): - for filename in files: - filename = os.path.relpath(os.path.join(root, filename), recipe_outdir) - files_list.append(os.path.join(destdir, filename)) finally: tinfoil.shutdown() From patchwork Mon Aug 21 16:07:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8578EEE49AC for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by mx.groups.io with SMTP id smtpd.web11.1052.1692634312720545660 for ; Mon, 21 Aug 2023 09:11:53 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=nYWq6D+i; spf=pass (domain: gmail.com, ip: 209.85.208.44, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-5259cf39154so4470346a12.2 for ; Mon, 21 Aug 2023 09:11:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634311; x=1693239111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YtK+g2G2EC9P2eSmH3IzPAwBYrV1YOaey925naodtdQ=; b=nYWq6D+i15WfjKIae3qSaN2okcfAICmSSiyOrULXjl0eqxxAUB+ntmXWY9MggLEZcf w9xAsltFN2KeQRcz8pPmrpE+bEwA9RoDz9Y+ns9vZsYH6zsaiD+oqz89vel/UZ+nMFwm pII0jgecT+Eco4AuTtbl/KKl6s+f/Y+/CvZSc1iOlqsUCzjxu/hU4DOpiz38NnoI6DFD xsm6RaNGvcQOh43beds8KsKLKBPwCvJvl9hCxd5xOZhDylcq+PbS4b2kJu56SRVo2NDF 97OH54Kgfqpy6aH7ukkts0xCQ4y1mDU3w35WZ8AqLO9YnJ7P+/DoP7QNv9cC4OJvFDSG MF3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634311; x=1693239111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YtK+g2G2EC9P2eSmH3IzPAwBYrV1YOaey925naodtdQ=; b=WslfZJt/HfjlAvzMgqbLO6J9CFnZkfemIueteB3T/lMUfDC2bCHWjpKYquhIoRXB4+ 9ZWVYoDC/fPyG70vHT47sk/qQUXQiLXEsrq92lorZaYSBe5IiNAg7CdWqz2cDcf7atQ5 IgOl/smdpMh11UXEYCHw8o4B1khm01D/dKvuUOMSsxie8WPkyhhTnpQHfUFVUeEp/0PP RWaNC1Y8xDNAujkP7YnsefH7QNuL01T8mkZVBXdA/hHn10RqwqtMaZPqOtED11/AFF9H +A7Qruj9HxENopPHMnWlxfqtGhUynQFlPgl4znw3GdKswRdWgWOjev7ydqaj2j4D7E3i XOSg== X-Gm-Message-State: AOJu0Ywt0SUhuM94efDeFdo2SpFam6LVnT+tQ2akuyksuBqKZjbZ6x2+ ViyqwrlMT9S24VRtvNOAft6KbkVyhOo= X-Google-Smtp-Source: AGHT+IEfMv1gj2aQqM9aFYUYPg0EnFw8AVl4j5TS14QOAdwUrXub0t8mM0zc9nz2t56AXCMS8kEdVw== X-Received: by 2002:aa7:c84b:0:b0:529:f97d:8870 with SMTP id g11-20020aa7c84b000000b00529f97d8870mr5015320edt.40.1692634310882; Mon, 21 Aug 2023 09:11:50 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:50 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 12/14] devtool: ide make deploy-target quicker Date: Mon, 21 Aug 2023 18:07:15 +0200 Message-ID: <20230821161120.171956-13-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186475 --- scripts/lib/devtool/ide.py | 115 ++++++++++++++++++------------------- 1 file changed, 55 insertions(+), 60 deletions(-) diff --git a/scripts/lib/devtool/ide.py b/scripts/lib/devtool/ide.py index 7c7040232c4..9c724509d48 100755 --- a/scripts/lib/devtool/ide.py +++ b/scripts/lib/devtool/ide.py @@ -9,6 +9,7 @@ import os import stat +import sys import logging import json import re @@ -360,15 +361,19 @@ class RecipeModified: self.bbappend = None # recipe variables from d.getVar self.b = None + self.base_libdir = None self.bpn = None self.d = None self.fakerootcmd = None self.fakerootenv = None + self.libdir = None + self.max_process = None self.package_arch = None self.path = None self.recipe_sysroot = None self.recipe_sysroot_native = None self.staging_incdir = None + self.strip_cmd = None self.target_arch = None self.workdir = None # replicate bitbake build environment @@ -387,11 +392,6 @@ class RecipeModified: self.meson_cross_file = None # vscode self.dot_code_dir = None - # TODO: remove calling devtool - self.bb_env_passthrough_additions = None - self.bbpath = None - self.bitbakepath = None - self.topdir = None def initialize(self, config, workspace, tinfoil): recipe_d = parse_recipe( @@ -413,10 +413,14 @@ class RecipeModified: shutil.rmtree(self.temp_dir) self.b = recipe_d.getVar('B') + self.base_libdir = recipe_d.getVar('base_libdir') self.bpn = recipe_d.getVar('BPN') self.d = recipe_d.getVar('D') self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD') self.fakerootenv = recipe_d.getVar('FAKEROOTENV') + self.libdir = recipe_d.getVar('libdir'), + self.max_process = int(recipe_d.getVar( + "BB_NUMBER_THREADS") or os.cpu_count() or 1) self.package_arch = recipe_d.getVar('PACKAGE_ARCH') self.path = recipe_d.getVar('PATH') self.recipe_sysroot = os.path.realpath( @@ -425,15 +429,10 @@ class RecipeModified: recipe_d.getVar('RECIPE_SYSROOT_NATIVE')) self.staging_incdir = os.path.realpath( recipe_d.getVar('STAGING_INCDIR')) + self.strip_cmd = recipe_d.getVar('STRIP') self.target_arch = recipe_d.getVar('TARGET_ARCH') self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR')) - self.bb_env_passthrough_additions = recipe_d.getVar( - 'BB_ENV_PASSTHROUGH_ADDITIONS') - self.bbpath = recipe_d.getVar('BBPATH') - self.bitbakepath = recipe_d.getVar('BITBAKEPATH') - self.topdir = recipe_d.getVar('TOPDIR') - self.__init_exported_variables(recipe_d) if bb.data.inherits_class('cmake', recipe_d): @@ -971,6 +970,38 @@ class RecipeModified: cmd_lines.append(install_cmd) return self.write_script(cmd_lines, 'bb_run_do_install') + def gen_deploy_target_script(self, args): + """Generate a quicker (works without tinfoil) variant of devtool target-deploy""" + cmd_lines = ['#!/usr/bin/env python3'] + cmd_lines.append('import sys') + cmd_lines.append('devtool_sys_path = %s' % str(sys.path)) + cmd_lines.append('devtool_sys_path.reverse()') + cmd_lines.append('for p in devtool_sys_path:') + cmd_lines.append(' if p not in sys.path:') + cmd_lines.append(' sys.path.insert(0, p)') + cmd_lines.append('from devtool.deploy import deploy_cached') + args_filter = ['debug', 'dry_run', 'key', 'no_check_space', 'no_host_check', + 'no_preserve', 'port', 'recipename', 'show_status', 'ssh_exec', 'strip', 'target'] + filtered_args_dict = {key: value for key, value in vars( + args).items() if key in args_filter} + cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict)) + cmd_lines.append('class Dict2Class(object):') + cmd_lines.append(' def __init__(self, my_dict):') + cmd_lines.append(' for key in my_dict:') + cmd_lines.append(' setattr(self, key, my_dict[key])') + cmd_lines.append('filtered_args = Dict2Class(filtered_args_dict)') + cmd_lines.append('deploy_cached("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args)' % + (self.d, self.workdir, self.path, self.strip_cmd, + self.libdir, self.base_libdir, self.max_process, + self.fakerootcmd, self.fakerootenv)) + return self.write_script(cmd_lines, 'deploy_target') + + def gen_install_deploy_script(self, args): + cmd_lines = ['#!/bin/sh'] + cmd_lines.append(self.gen_fakeroot_install_script()) + cmd_lines.append(self.gen_deploy_target_script(args)) + return self.write_script(cmd_lines, 'install_and_deploy') + def write_script(self, cmd_lines, script_name): bb.utils.mkdirhier(self.temp_dir) script_file = os.path.join(self.temp_dir, script_name) @@ -981,61 +1012,15 @@ class RecipeModified: return script_file def vscode_tasks(self, args, gdb_cross): - run_do_install = self.gen_fakeroot_install_script() - pythonpath_new = os.path.realpath( - os.path.join(self.bitbakepath, '../lib')) - try: - pythonpath = os.environ['PYTONPATH'].split(':') - if pythonpath_new not in pythonpath: - pythonpath = pythonpath_new + ":" + pythonpath - except KeyError: - pythonpath = pythonpath_new - install_label = "do_install %s" % self.bpn - deploy_label = "devtool deploy-target %s" % self.bpn - bb_env = { - # "BBPATH": self.bbpath, - "PYTHONPATH": pythonpath, - "BUILDDIR": self.topdir, - # Use e.g. Python from host, not python-native - "PATH": os.environ['PATH'], - "BB_ENV_PASSTHROUGH_ADDITIONS": self.bb_env_passthrough_additions - } - bb_options = { - "cwd": self.topdir - } + run_install_deploy = self.gen_install_deploy_script(args) tasks_dict = { "version": "2.0.0", "tasks": [ { - "label": install_label, - "type": "shell", - "command": run_do_install, - "problemMatcher": [] - }, - { - "label": deploy_label, + "label": "install && deploy-target %s" % self.bpn, "type": "shell", - "command": "devtool", # TODO: Generate a self contained script which does what devtool target-deploy does but without connection to the bitbake server - "args": ["--bbpath", self.bbpath, "deploy-target", self.bpn, args.target, "--strip", "--no-host-check"], - "linux": { - "options": { - "env": bb_env - }}, - "options": bb_options, + "command": run_install_deploy, "problemMatcher": [] - }, - { - "label": "install && deploy-target %s" % self.bpn, - "dependsOrder": "sequence", - "dependsOn": [ - install_label, - deploy_label - ], - "problemMatcher": [], - "group": { - "kind": "build", - "isDefault": True - } } ] } @@ -1183,6 +1168,16 @@ def register_commands(subparsers, context): '--skip-bitbake', help='Generate IDE configuration but skip calling bibtake to update the SDK.', action='store_true') parser_ide.add_argument( '-k', '--bitbake-k', help='Pass -k parameter to bitbake', action='store_true') + parser_ide.add_argument( + '--no-strip', help='Do not strip executables prior to deploy', dest='strip', action='store_false') + parser_ide.add_argument( + '-n', '--dry-run', help='List files to be undeployed only', action='store_true') + parser_ide.add_argument( + '-s', '--show-status', help='Show progress/status output', action='store_true') + parser_ide.add_argument( + '-p', '--no-preserve', help='Do not preserve existing files', action='store_true') + parser_ide.add_argument( + '--no-check-space', help='Do not check for available space before deploying', action='store_true') parser_ide.add_argument( '--debug-build-config', help='Use debug build flags, for example set CMAKE_BUILD_TYPE=Debug', action='store_true') parser_ide.set_defaults(func=ide_setup) From patchwork Mon Aug 21 16:07:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D969EE49A6 for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by mx.groups.io with SMTP id smtpd.web11.1053.1692634313639824991 for ; Mon, 21 Aug 2023 09:11:53 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=m2OUsodB; spf=pass (domain: gmail.com, ip: 209.85.218.54, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-99df11828c6so963999166b.1 for ; Mon, 21 Aug 2023 09:11:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634312; x=1693239112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=l4lyq43eF+4jjlYBVgBi6SbESThlcXAZT+y0qQ1cgyE=; b=m2OUsodBR3nbEOAPWRl2w5gBa9PyhCPjBKCXgbGR//0gkZmPvrc9f0hvxon/ySvrVV eVZc1S1He1M87siOE5uPKCtZNi9g9lNxJTfItPhb2qtRndzuSOTKKurRqJZ6sXjWEIe5 yQJWCvr441xEkA1PkOKxnBeK504pmaUqFtFKWzsRUrVS+iqDCNKG9ulVygqRYbyjXUwZ 7nrsVsANlsP5yaFiTY7Nkb5TqWMv9HbGMoNLhWMvPy2odOHV6IT2qHkQcmMp0pqTAxpJ D/+QakNlOgVlMcshs0gMeKwurGVOjhDOKEX3RpvGJ7bulwzl6rYaHmyweljyZmFoA4pR m/1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634312; x=1693239112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l4lyq43eF+4jjlYBVgBi6SbESThlcXAZT+y0qQ1cgyE=; b=QVcIqDwqzqABC8FTxbhAdXuhY4z1a7lx0aSY+KY9otCN01JueWWdA+2ER9ueScqyHB GPYAdyb661ygdx/sBBeMII+1Y2wECjwD3fYo8Teth+9s4nxIFVUBLuxVI0qrbopFsgXF 5/wXLih0Lk4UGeozdjkri7aLKkL6R04iVMobOv0RpEYQzwQ6voOSye6cSL8LyAGazvtr /fOPbEniYzbisZehy7FZqbHHEoPD7i6cXq+2ReELH0gdclff+DKoOPQnMGmONEbL/684 ZBwLWL58ND4ktfS943BIlz4W1tc0v0hHF5WEohqbWYLZgE7Li2FgI78hAWb5eDYRg1nV Ovig== X-Gm-Message-State: AOJu0YxCBadBGqHy/TwU0na8zUBbF6CAPuiCACRDteQ+F8m8T2GZ/AWf AMTMIKsCqBlSDV285XC8R77LB79Riko= X-Google-Smtp-Source: AGHT+IEbMPQ1o9fvtEgDauHGFjdcR1H53T37t1gH42LdNm75jwVIwGLyPXWuBrY1V7Is5jZBfGzjbg== X-Received: by 2002:a17:907:9483:b0:99d:f274:316e with SMTP id dm3-20020a170907948300b0099df274316emr6722309ejc.2.1692634311803; Mon, 21 Aug 2023 09:11:51 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:51 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 13/14] oe-selftest devtool: ide tests Date: Mon, 21 Aug 2023 18:07:16 +0200 Message-ID: <20230821161120.171956-14-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186476 Signed-off-by: Adrian Freihofer --- meta/lib/oeqa/selftest/cases/devtool.py | 133 ++++++++++++++++++++++++ 1 file changed, 133 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 14a80d5ff4c..083ecd4acfb 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -11,6 +11,7 @@ import tempfile import glob import fnmatch import unittest +import json from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer @@ -2192,3 +2193,135 @@ class DevtoolUpgradeTests(DevtoolBase): #Step 4.5 runCmd("grep %s %s" % (modconfopt, codeconfigfile)) + + +class DevtoolIdeTests(DevtoolBase): + + def __devtool_ide_recipe(self): + tempdir = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir) + self.track_for_cleanup(self.workspacedir) + self.add_command_to_tearDown('bitbake -c clean %s' % self.recipe_name) + self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') + + conf_lines = [ + 'IMAGE_FEATURES += "ssh-server-openssh"', + 'IMAGE_CLASSES += "image-combined-dbg"', + 'IMAGE_GEN_DEBUGFS = "1"', + 'IMAGE_INSTALL += "gdbserver %s-ptest"' % self.recipe_name + ] + self.write_config("\n".join(conf_lines)) + + result = runCmd('devtool modify %s -x %s' % (self.recipe_name, tempdir)) + self.assertExists(os.path.join(tempdir, self.build_file), + 'Extracted source could not be found') + self.assertExists(os.path.join(self.workspacedir, 'conf', + 'layer.conf'), 'Workspace directory not created') + matches = glob.glob(os.path.join(self.workspacedir, + 'appends', self.recipe_name + '.bbappend')) + self.assertTrue(matches, 'bbappend not created %s' % result.output) + + # Test devtool status + result = runCmd('devtool status') + self.assertIn(self.recipe_name, result.output) + self.assertIn(tempdir, result.output) + self._check_src_repo(tempdir) + + result = runCmd('devtool ide %s core-image-minimal -c' % self.recipe_name) + return tempdir + + def __devtool_ide_qemu(self): + # Verify do_install works + with open(os.path.join(self.tempdir, '.vscode', 'tasks.json')) as tasks_j: + tasks_d = json.load(tasks_j) + tasks = tasks_d["tasks"] + task_install = next((task for task in tasks if task["label"] == "install && deploy-target %s" % self.recipe_name), None) + self.assertIsNot(task_install, None) + install_deploy_cmd = task_install["command"] + # execute only the bb_run_do_install script since the deploy would require e.g. Qemu running. + install_cmd = install_deploy_cmd.replace("install_and_deploy", "bb_run_do_install") + runCmd(install_cmd, cwd=self.tempdir) + + # Verify if re-building the SDK still works and the deploy and install script gets generated + runCmd('devtool ide %s core-image-minimal -c' % self.recipe_name) + self.assertExists(install_deploy_cmd, 'install_and_deploy script not found') + + # Verify deployment to Qemu works + with runqemu("core-image-minimal", runqemuparams = "nographic") as qemu: + MAGIC_STRING_ORIG = "Magic: 123456789" + MAGIC_STRING_NEW = "Magic: 987654321" + + # Verify the unmodified example prints the magic string + status, output = qemu.run(self.example_exe) + self.assertEqual(status, 0) + self.assertIn(MAGIC_STRING_ORIG, output) + + # Verify the unmodified ptests work + status, output = qemu.run("ptest-runner " + self.recipe_name) + self.assertEqual(status, 0, msg=output) + self.assertIn("PASS: cpp-example-lib", output) + + # Replace the Magic String in the code, compile and deploy to Qemu + cpp_example_lib_hpp = os.path.join(self.tempdir, 'cpp-example-lib.hpp') + with open(cpp_example_lib_hpp, 'r') as file: + cpp_code = file.read() + cpp_code = cpp_code.replace(MAGIC_STRING_ORIG, MAGIC_STRING_NEW) + with open(cpp_example_lib_hpp, 'w') as file: + file.write(cpp_code) + runCmd(install_deploy_cmd, cwd=self.tempdir) + + # Verify the modified example prints the modified magic string + status, output = qemu.run(self.example_exe) + self.assertEqual(status, 0) + self.assertNotIn(MAGIC_STRING_ORIG, output) + self.assertIn(MAGIC_STRING_NEW, output) + + # Verify the modified example ptests work + status, output = qemu.run("ptest-runner " + self.recipe_name) + self.assertEqual(status, 0, msg=output) + self.assertIn("PASS: cpp-example-lib", output) + + def test_devtool_ide_recipe_cmake(self): + self.recipe_name = "cmake-example" + self.example_exe = "cmake-example" + self.build_file = "CMakeLists.txt" + self.tempdir = self.__devtool_ide_recipe() + + with open(os.path.join(self.tempdir, 'CMakeUserPresets.json')) as cmake_preset_j: + cmake_preset_d = json.load(cmake_preset_j) + config_presets = cmake_preset_d["configurePresets"] + self.assertEqual(len(config_presets), 1) + cmake_exe = config_presets[0]["cmakeExecutable"] + preset_name = config_presets[0]["name"] + + # Verify the cmake preset generated by devtool ide is available + result = runCmd('%s --list-presets' % cmake_exe, cwd=self.tempdir) + self.assertIn(preset_name, result.output) + + # Verify cmake re-uses the o files compiled by bitbake + result = runCmd('%s --build --preset %s' % (cmake_exe, preset_name), cwd=self.tempdir) + self.assertIn("ninja: no work to do.", result.output) + + # Verify the unit tests work (in Qemu) + result = runCmd('%s --build --preset %s --target test' % (cmake_exe, preset_name), cwd=self.tempdir) + self.assertIn("100% tests passed", result.output) + + # Verify re-building and testing works again + result = runCmd('%s --build --preset %s --target clean' % (cmake_exe, preset_name), cwd=self.tempdir) + self.assertIn("Cleaning all built files...", result.output) + result = runCmd('%s --build --preset %s' % (cmake_exe, preset_name), cwd=self.tempdir) + self.assertIn("Building", result.output) + self.assertIn("Linking", result.output) + result = runCmd('%s --build --preset %s --target test' % (cmake_exe, preset_name), cwd=self.tempdir) + self.assertIn("Running tests...", result.output) + self.assertIn("100% tests passed", result.output) + + self.__devtool_ide_qemu() + + def test_devtool_ide_recipe_meson(self): + self.recipe_name = "meson-example" + self.example_exe = "mesonex" + self.build_file = "meson.build" + self.tempdir = self.__devtool_ide_recipe() + + self.__devtool_ide_qemu() From patchwork Mon Aug 21 16:07:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Freihofer X-Patchwork-Id: 29239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F7CBEE49AB for ; Mon, 21 Aug 2023 16:11:57 +0000 (UTC) Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by mx.groups.io with SMTP id smtpd.web11.1055.1692634314392561880 for ; Mon, 21 Aug 2023 09:11:54 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20221208 header.b=rOGVbCOT; spf=pass (domain: gmail.com, ip: 209.85.208.45, mailfrom: adrian.freihofer@gmail.com) Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5259cf39154so4470389a12.2 for ; Mon, 21 Aug 2023 09:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692634312; x=1693239112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=D+mSuVh7iCmEtFdHyUBXG4r59rQe0KW4Z0rsULAB9NU=; b=rOGVbCOT3inGcfgDjmtlubhNCgcuDa5cY4l6q+izppkxeYifWtxf7oh9AoVzFyFQ8X 4j2tZlzwF/sugJADOnW248/k2+p7DnBXh3zQBSBC5XLXrhKnK87ThycdP8Ubk83Pk6P2 gEntHqZSgZvmB9CAgGVCNWBy8fgWbmm2Md/PTtBewzyAg90w6rwf7kzgJ89WDUlzPbuD Z5RMbpfvZNLsoiRfU/BsCWC2Pe+DfnFRp2TBtXgD899nteE9uSxQGTaQYo6RtN7Xviut cA+3V4CeZawLC5nUVEO8alRWQOfbRyV28BVfNGmY9s9tBrynJUhSk1jxh0VgD1tefrZF G1Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634312; x=1693239112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D+mSuVh7iCmEtFdHyUBXG4r59rQe0KW4Z0rsULAB9NU=; b=MBxNUgLnuda8255WwuuEOjVfUyz/pEVkOw+OOqntwyn+o7TCCu3KkJ0+tXq49dQdN7 m6Zzv6F+8CYjNcEMN014QkIXkwx/bw+KnQRwUix/IwrgssOmclF1Fa++s2z1LGuJEMhN 1aWe9GRZEwOpx3LLp8+VwHQYrWErr1/jNNAKGpG8cwoXG2VGiUQ4AVCQblKo9uGGIlbP TcecYs6+yi5dJIeguAV6k7AJJckp0F2JdeDIKvxBhDi816Tyu9/vOnPFM8JjPTSXxYVX IV0w7jLYC36rOqFIrF9R1gKUmhRCOvyqSBdpag0wjbAIopAfPDbnG/KwtEPBYlK9uNpw 8jEA== X-Gm-Message-State: AOJu0Yz8HkI7lMpHIMwJ3EngudD2ilVRZwUFXiAS41D4YD/uK4ENdfpq XqVqcDINuyGN6yATmNsFKAuqQ/HDhpU= X-Google-Smtp-Source: AGHT+IFSRNWYcUOlvPmiBlNjMetyfyMB/J8prz+VVqicnRomuYF/wxDJQpxHoDmv0dmlN0Y2OqFgVQ== X-Received: by 2002:aa7:d4da:0:b0:522:40dd:74f3 with SMTP id t26-20020aa7d4da000000b0052240dd74f3mr5041019edr.39.1692634312609; Mon, 21 Aug 2023 09:11:52 -0700 (PDT) Received: from t14s-af._sites.dc._msdcs.ad007.siemens.net ([2a02:169:59a6:0:5488:f785:9061:cf6c]) by smtp.gmail.com with ESMTPSA id x21-20020aa7dad5000000b0052284228e3bsm6175579eds.8.2023.08.21.09.11.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:11:52 -0700 (PDT) From: Adrian Freihofer X-Google-Original-From: Adrian Freihofer To: openembedded-core@lists.openembedded.org Cc: Adrian Freihofer Subject: [PATCH 14/14] docs: cover devtool ide Date: Mon, 21 Aug 2023 18:07:17 +0200 Message-ID: <20230821161120.171956-15-adrian.freihofer@siemens.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230821161120.171956-1-adrian.freihofer@siemens.com> References: <20230821161120.171956-1-adrian.freihofer@siemens.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 21 Aug 2023 16:11:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186477 Signed-off-by: Adrian Freihofer --- documentation/sdk-manual/extensible.rst | 100 +++++++++++++++++++++++- 1 file changed, 99 insertions(+), 1 deletion(-) diff --git a/documentation/sdk-manual/extensible.rst b/documentation/sdk-manual/extensible.rst index 9e08e57a4e7..81a46c4767f 100644 --- a/documentation/sdk-manual/extensible.rst +++ b/documentation/sdk-manual/extensible.rst @@ -237,7 +237,7 @@ all the commands. devtool quick reference. -Three ``devtool`` subcommands provide entry-points into +Four ``devtool`` subcommands provide entry-points into development: - *devtool add*: Assists in adding new software to be built. @@ -245,6 +245,8 @@ development: - *devtool modify*: Sets up an environment to enable you to modify the source of an existing component. +- *devtool ide*: Generates a configuration for an IDE. + - *devtool upgrade*: Updates an existing recipe so that you can build it for an updated set of source files. @@ -632,6 +634,102 @@ command: proceed with your work. If you do use this command, realize that the source tree is preserved. +Use ``devtool ide`` to generate an configuration for the IDE +------------------------------------------------------------ + +``devtool ide`` automatically configures the IDE for cross-compiling and remote debugging. +The IDE is configured to call for example cmake directly. This has several advantages. +First of all it is much faster than using e.g. ``devtool build``. But it also allows to +use the very good integration of build tools like cmake or gdb with VSCode directly. + +Two different use cases are supported: + +- Generate the IDE configuration for a workspace created by ``devtool modify``. + +- Generate the IDE configuration for using a cross-toolchain as provided by + ``bitbake meta-ide-support build-sysroots``. + +Assuming the development environment is set up correctly and a workspace has been created +for the recipe using ``devtool modify recipe``, the following command can create the +configuration for VSCode in the recipe workspace: + + $ devtool ide recipe core-image-minimal --target root@192.168.7.2 + +What this command does exactly depends on the recipe or the build tool used by the recipe. +Currently, only CMake and Meson are supported natively. + +For a recipe which inherits cmake it does: + +- Prepare the SDK by calling bitbake core-image-minimal, gdb-cross, qemu-native... + +- Generate a cmake-preset with configures cmake to use exactly the same environent and + the same cmake-cache configuration as used by ``bitbake recipe``. The cmake-preset referres + to the per-recipe-sysroot of the recipe. + + Currently Configure, Build and Test presets are supported. Test presets execute the test + binaries with Qemu. + +- Generates a helper script to handle the do_install with pseudo + +- Generates some helper scripts to start the gdbserver on the target device + +- Generates the ``.vscode`` folder containing the following files: + + - c_ccp_properties.json: configure the code navigation + + - extensions.json: Recommend the extensions which are used. + + - launch.json: Provide a configuration for remote debugging with gdb-cross and gdbserver. + The debug-symbols are searched in the build-folder, the per-recipe-sysroot and the rootfs-dbg + folder which is provided by the image. + + - settings.json: confgure the indexer to ignore the build folders + + - tasks.json: Provide some helpers for running + + - do_install and ``devtool deploy-target`` + + - start the gdbserver via ssh + +For a recipe which inherits meson a similar configuration is generated. +Because there is nothing like a meson-preset a wrapper script for meson is generated. + +If there is no native support by ``devtool ide`` for the build tool used by the recipe a generic +IDE configuration calling ``devtool build recipe`` is generated. + +For some special recipes and use cases a per-recipe-sysroot based SDK is not suitable. +Therefore devtool ide also supports setting up the shared sysroots environment and generating +a IDE configurations referring to the shared sysroots. Recipes leading to a shared sysroot +are for example meta-ide-support or shared-sysroots. Also passing none as a recipe name leads +to a shared sysroot SDK. + + $ devtool ide none core-image-minimal + +In case of a shared sysroot SDK the configuration which gets generated for VSCode exposes the +cross-tool-chain as a cmake-kit. If a cmake project is loaded into VSCode the cross-toolchain +can be selected for compiling. + +The default IDE is VSCode. Some hints about using VSCode: + +- To work with cmake press ``Ctrl + Shift + p``, type cmake. + This will show some possible commands like selecting a cmake preset, compiling or running ctest. + A cmake kit might be activated by ``Ctrl + Shift + p``, type cmake quick start, + if not preset file is in the wokspace. + +- To work with meson press ``Ctrl + Shift + p``, type meson. + This will show some possible commands like compiling or executing the unit tests. + +- For the deployment to the target device, just press ``Ctrl + Shift + p``, type task. + Select the install & deploy task. + +- For remote debugging, switch to the debugging view by pressing the play button with the bug on the left side. + This will provide a green play button with a drop-down list where a debug configuration can be selected. + After selecting one of the generated configurations, press the play button. + +Additionally ``--ide=none`` is supported. +With the none IDE some generic configurations files like .gdbinit files and some helper scripts +are generated. + Use ``devtool upgrade`` to Create a Version of the Recipe that Supports a Newer Version of the Software -------------------------------------------------------------------------------------------------------