From patchwork Fri Nov 3 10:28:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kanavin X-Patchwork-Id: 33543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B80F6C4167D for ; Fri, 3 Nov 2023 10:28:14 +0000 (UTC) Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by mx.groups.io with SMTP id smtpd.web10.48888.1699007292451637914 for ; Fri, 03 Nov 2023 03:28:12 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GCFX+mQz; spf=pass (domain: gmail.com, ip: 209.85.128.41, mailfrom: alex.kanavin@gmail.com) Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-40891d38e3fso13570315e9.1 for ; Fri, 03 Nov 2023 03:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699007291; x=1699612091; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=FufGndjz5XVITAICk+pt8iy4iNJuEI8HmhN8/wwcmLU=; b=GCFX+mQzzOuBF3xCJwN0JGw1FgYutOTbLfaS8rst4ksBgxGuwDbxwoa0jt8rpJE5s1 ZBV6xw0Dgm+BBU5CT4hfiI2eSTrsxZhANT4xMgod1A3u202HYnAc+E36HfRo4LnpAhZK VHabtnRyH0vlz/+Ty94uZoQr/T0/YqZJDasy8SiRaYATM5luBX+MBQcuqt70hKOzJqjP 04Ri98ftBoemMxDgd734ZjL5cgSM+gsEL/uFfncggCz4yXleE6FJO0HG/iuXAXAxTgUo Mc/1meQjtgjK5Ea1TeJSF1OtS8hXqLO6M6VfEqERJUf0qJge9qPXbjCnz7Aq06hDSk5h 0mbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699007291; x=1699612091; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FufGndjz5XVITAICk+pt8iy4iNJuEI8HmhN8/wwcmLU=; b=Y2RA/x/ZszYZSwnnf4Ji6DMovgr+SZC8hNbnY3BBstPrdYq4/H9fnOmYBBsYYNe6E2 MSM1OQrguntBITvsaGP8G7ivxCRQ9KBXdqpFu9wiuysxxfOK/fYTikzgZlHdTSsDCPeF 97sOM4XAk9YbN20GGtQKCLJPXgxqBwhPvMJWZS4xfkB7V1Q5phq340nFt3DFavu1bSwQ w8pE+LL7enRtGVPAtb8YD5Sd+raBHudSQagsaqvvuokxFDMgEOSAmZQfXX+CKNq7RRMA PlSb6AHur0EvIv4OBfC0SEzou3RoKiC+KcQXqErAjYce8gv9rBf02l9snRYMUdojh7tl cmUA== X-Gm-Message-State: AOJu0Yye9Pk4QDPxLtCSXAB0SIdXqKODaKHZeKjofz+nJ5TPY2U6XpX4 1YeJ6P96mIH0Zao4Cnx9lTnChWr2Dvg= X-Google-Smtp-Source: AGHT+IGS0wMmQUZ6QXl+b0piepD8EycfzmHry4bZxZaDsHSsMRJhcEMToVM5S8ikIcCXzYd2ZlL3CQ== X-Received: by 2002:a05:600c:1991:b0:405:7400:1e3d with SMTP id t17-20020a05600c199100b0040574001e3dmr17219206wmq.32.1699007290447; Fri, 03 Nov 2023 03:28:10 -0700 (PDT) Received: from Zen2.lab.linutronix.de. (drugstore.linutronix.de. [80.153.143.164]) by smtp.gmail.com with ESMTPSA id y14-20020a05600c2b0e00b00405718cbeadsm1976643wme.1.2023.11.03.03.28.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 03:28:10 -0700 (PDT) From: Alexander Kanavin X-Google-Original-From: Alexander Kanavin To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin Subject: [PATCH 1/4] populate_sdk_ext.bbclass: do not symlink unfsd from sdk image sysroot into eSDK tools path Date: Fri, 3 Nov 2023 11:28:03 +0100 Message-Id: <20231103102806.2332746-1-alex@linutronix.de> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 03 Nov 2023 10:28:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/190148 This was done in 2016 to support Eclipse plugin (long dead), it's currently broken as image sysroot is not in the SDK until the image is built in that context, and current tools all rely on runqemu-export-rootfs which does not rely on PATH and runs unfsd with full path to recipe-specific sysroots. Signed-off-by: Alexander Kanavin --- meta/classes-recipe/populate_sdk_ext.bbclass | 7 ------- 1 file changed, 7 deletions(-) diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass index 028d0c1ff54..173b3065b00 100644 --- a/meta/classes-recipe/populate_sdk_ext.bbclass +++ b/meta/classes-recipe/populate_sdk_ext.bbclass @@ -635,13 +635,6 @@ install_tools() { test -e ${targetscriptfn} || ln -rs ${scriptfn} ${targetscriptfn} done done - # We can't use the same method as above because files in the sysroot won't exist at this point - # (they get populated from sstate on installation) - unfsd_path="${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/unfsd" - if [ "${SDK_INCLUDE_TOOLCHAIN}" = "1" -a ! -e $unfsd_path ] ; then - binrelpath=${@os.path.relpath(d.getVar('STAGING_BINDIR_NATIVE'), d.getVar('TMPDIR'))} - ln -rs ${SDK_OUTPUT}/${SDKPATH}/tmp/$binrelpath/unfsd $unfsd_path - fi touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase # find latest buildtools-tarball and install it From patchwork Fri Nov 3 10:28:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kanavin X-Patchwork-Id: 33542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDAC6C4167B for ; Fri, 3 Nov 2023 10:28:14 +0000 (UTC) Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by mx.groups.io with SMTP id smtpd.web11.48041.1699007292764027802 for ; Fri, 03 Nov 2023 03:28:13 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=gd5GVww8; spf=pass (domain: gmail.com, ip: 209.85.128.51, mailfrom: alex.kanavin@gmail.com) Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-40853c639abso14206575e9.0 for ; Fri, 03 Nov 2023 03:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699007291; x=1699612091; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ieahQZSj6qKJmQKsOaGmi7ikLiTlObdwxrLsrN6IrcQ=; b=gd5GVww8Gym7u9QdaX/yvZ+Nzq3oMSVgyNw01oDWsjp6HhTpxN0d4RWmFsSwF9BpO2 53UHT0ZS7JBBoG0EtWlUIOCtV2ktvsfDf6RU0Fwe43FUMl8E2wTRonvBwz4u3bsUFbd4 e7Auevrd1KPs6lh0QjNw0iidNqfAEOABH/wsbwU3RzlU/tpuupdGnMIPMgYU87MI+Vrw v9S3+BHB08pJ07k+5PEb8kA6ZEeVwlut+RkUbF4T2x2w883h0bCiGXDDlf92CXFVxwW/ RSO4bKLTKcFW1QP5YQSyhpg0jmL22P4oytK6bME4zfQNAn40unmALU8s0uHSJxqfTTEG DZug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699007291; x=1699612091; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ieahQZSj6qKJmQKsOaGmi7ikLiTlObdwxrLsrN6IrcQ=; b=I++2IQxr+nfd335P00NhoQ54EhsHDJX4aZVL5p1aYpVGJYigjumc5AjW/+CgtjVGFV b9NAjPvXV+xSU+8V+I4nSazGRxJ1pPuL+ImoA9SKYf9GbXAIkIwRvWcG4YjCT7JaiTdp WkypBnJggOfxqFp3Qp5P1ffnz3oRp/96dDeEc53jxgZkJaEnA5ZIUCsOXlTGKBj2Q+g9 W7MYINLaqaEXyxLkvk4HtZLW/bTp4ALPdJXUA9cDRfF4ahWx7ua1m1gcsJ6d+TYfopRY DAGY0LoooMMG0mho7lfDFqEl7uUw4ffMRtNu3OZBrU9uTvkMrkWzzkHv2+E58Q/o8vF3 UN2g== X-Gm-Message-State: AOJu0YzZdxMidH9ifq7jtG0XJU4svqKb+XYtE7Qgvkrn9KNqOdip4Ak6 AiZIRd+imuA8M0DT3rpTajOjLU7xKSY= X-Google-Smtp-Source: AGHT+IGqVSx+wO0fucOkqjDYBYbizbc9IWD4SLDcvxhkSc2BJPRUMuQh5RiG8FvH6fJdlTFfdI4rVQ== X-Received: by 2002:a05:600c:46cf:b0:406:52f1:7e6f with SMTP id q15-20020a05600c46cf00b0040652f17e6fmr18531362wmo.12.1699007290934; Fri, 03 Nov 2023 03:28:10 -0700 (PDT) Received: from Zen2.lab.linutronix.de. (drugstore.linutronix.de. [80.153.143.164]) by smtp.gmail.com with ESMTPSA id y14-20020a05600c2b0e00b00405718cbeadsm1976643wme.1.2023.11.03.03.28.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 03:28:10 -0700 (PDT) From: Alexander Kanavin X-Google-Original-From: Alexander Kanavin To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin Subject: [PATCH 2/4] meta/lib/oe/copy_buildsystem.py: do not derefence symlinks Date: Fri, 3 Nov 2023 11:28:04 +0100 Message-Id: <20231103102806.2332746-2-alex@linutronix.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231103102806.2332746-1-alex@linutronix.de> References: <20231103102806.2332746-1-alex@linutronix.de> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 03 Nov 2023 10:28:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/190149 This was added (I think) for the purpose of supporting layers that refer to items outside of the layer via relative symlinks: https://git.yoctoproject.org/poky-contrib/commit/?id=d31d1ad4e566e42d0bbcf1f41ac25e33181fb517 I do not think copying the link target into the layer that references it is the correct solution: rather the original target should be included into the SDK with the same relative path. This change is done for the sake of preserving symlinks that are referencing things inside the layer as they are; particularly the content of scripts/esdk-tools/. Signed-off-by: Alexander Kanavin --- meta/lib/oe/copy_buildsystem.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py index a0d829054e1..81abfbf9e23 100644 --- a/meta/lib/oe/copy_buildsystem.py +++ b/meta/lib/oe/copy_buildsystem.py @@ -22,7 +22,7 @@ def _smart_copy(src, dest): mode = os.stat(src).st_mode if stat.S_ISDIR(mode): bb.utils.mkdirhier(dest) - cmd = "tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C %s -p . \ + cmd = "tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -cf - -C %s -p . \ | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dest) subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT) else: From patchwork Fri Nov 3 10:28:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kanavin X-Patchwork-Id: 33545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B80B0C4332F for ; Fri, 3 Nov 2023 10:28:14 +0000 (UTC) Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by mx.groups.io with SMTP id smtpd.web11.48042.1699007293085856591 for ; Fri, 03 Nov 2023 03:28:13 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Lk2Pn4XO; spf=pass (domain: gmail.com, ip: 209.85.128.54, mailfrom: alex.kanavin@gmail.com) Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-40837ebba42so12257035e9.0 for ; Fri, 03 Nov 2023 03:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699007291; x=1699612091; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JuFfbLlDjuZ6y9ZPhHyTkBIvvQUm/cHH890IM4H/3cw=; b=Lk2Pn4XOgMKdwin0/mqEWKw6w++1dL0VqiHXIhoXBPBDtsmaEOM4RqGnNkFd1qq/b8 +oWjrDj/cS8frjUNHjwpxumwtvxM84+ilpVavgIWyK1SLKxS3qoaLp3n3bcfsyOKQiP3 QyuCKuIFzvbpkiOWKLpE+g+9x5hrLwbKb3dm9Y/uMJDj/hqFBjHzqACLgtusL7cCknvu E8dRpSvKffFRFfpcEExvZ/NIrBDy7fGxosa1K/Sep75Fk/Ab/PMMLXkfbALqJwakETwp aoMJKECkpyqOCBmMykoHFi/rl7ANEYrfpOa9utmZFMQXC6VyIC6BrvFvhI22JJyW2qoQ L3bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699007291; x=1699612091; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JuFfbLlDjuZ6y9ZPhHyTkBIvvQUm/cHH890IM4H/3cw=; b=LOEhHF5XTjnwnn3ofbQqQVEV3y1kraBgEMeaYhJpnWHuFLOsCJZsFwZW7vsfRZ6HI0 VIa7fqPY0o85v15bPZjNiBC1q+EEeVNdT3rU7rh/s9XSaOdG12QESs3GVT0FfD1euF3M DdhNVHHa12PqSYYhdNjMXoTfLnCn6ymUAcLYGzfWmxTqBRePEOiiTM0xWsWxk/HKoTNF /BsDqGL/F5oshIlLXwN/L/EesjaG/4hG/z2+3aANobkPFR7jeHYaGJbCbTxDtEqib8Wd lxZFoXfLbtPzIRv2dncF/T+oVY+KtlvtgS00z9Enw84zZtLVJrTnXQaU0cXjClzmuEgZ SjaA== X-Gm-Message-State: AOJu0Yw6VsxrlUwQjqMWJYmRmJZCIsRUjmaEqzKcyc4rqY+XM8yvAXfB lscZXAPVOSgrbZaRDDAMtm1p318Yl1o= X-Google-Smtp-Source: AGHT+IFrJjPUD3PCw3zE1hlj4gl/kqQXJRQx9gQ9hwNCcobZmcn08n48GN+zMSW3RVWkxDD6fXBbbA== X-Received: by 2002:a05:600c:4f91:b0:408:4120:bad2 with SMTP id n17-20020a05600c4f9100b004084120bad2mr18819558wmq.9.1699007291435; Fri, 03 Nov 2023 03:28:11 -0700 (PDT) Received: from Zen2.lab.linutronix.de. (drugstore.linutronix.de. [80.153.143.164]) by smtp.gmail.com with ESMTPSA id y14-20020a05600c2b0e00b00405718cbeadsm1976643wme.1.2023.11.03.03.28.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 03:28:11 -0700 (PDT) From: Alexander Kanavin X-Google-Original-From: Alexander Kanavin To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin Subject: [PATCH 3/4] scripts/esdk-tools: use a dedicated, static directory for esdk tools Date: Fri, 3 Nov 2023 11:28:05 +0100 Message-Id: <20231103102806.2332746-3-alex@linutronix.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231103102806.2332746-1-alex@linutronix.de> References: <20231103102806.2332746-1-alex@linutronix.de> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 03 Nov 2023 10:28:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/190150 This allows easier replication of esdk environment (which provides a curated, limited set of tools that for example does not include bitbake) in a standard yocto build. Switchover between various sets can be achieved via PATH manipulation. Signed-off-by: Alexander Kanavin --- meta/classes-recipe/populate_sdk_ext.bbclass | 16 ++++------------ scripts/esdk-tools/devtool | 1 + scripts/esdk-tools/oe-find-native-sysroot | 1 + scripts/esdk-tools/recipetool | 1 + scripts/esdk-tools/runqemu | 1 + scripts/esdk-tools/runqemu-addptable2image | 1 + scripts/esdk-tools/runqemu-export-rootfs | 1 + scripts/esdk-tools/runqemu-extract-sdk | 1 + scripts/esdk-tools/runqemu-gen-tapdevs | 1 + scripts/esdk-tools/runqemu-ifdown | 1 + scripts/esdk-tools/runqemu-ifup | 1 + scripts/esdk-tools/wic | 1 + 12 files changed, 15 insertions(+), 12 deletions(-) create mode 120000 scripts/esdk-tools/devtool create mode 120000 scripts/esdk-tools/oe-find-native-sysroot create mode 120000 scripts/esdk-tools/recipetool create mode 120000 scripts/esdk-tools/runqemu create mode 120000 scripts/esdk-tools/runqemu-addptable2image create mode 120000 scripts/esdk-tools/runqemu-export-rootfs create mode 120000 scripts/esdk-tools/runqemu-extract-sdk create mode 120000 scripts/esdk-tools/runqemu-gen-tapdevs create mode 120000 scripts/esdk-tools/runqemu-ifdown create mode 120000 scripts/esdk-tools/runqemu-ifup create mode 120000 scripts/esdk-tools/wic diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass index 173b3065b00..53adc868d47 100644 --- a/meta/classes-recipe/populate_sdk_ext.bbclass +++ b/meta/classes-recipe/populate_sdk_ext.bbclass @@ -220,16 +220,16 @@ python copy_buildsystem () { if os.path.exists(os.path.join(baseoutpath, relpath)): conf_initpath = relpath - relpath = os.path.join('layers', path, 'scripts', 'devtool') + relpath = os.path.join('layers', path, 'scripts', 'esdk-tools', 'devtool') if os.path.exists(os.path.join(baseoutpath, relpath)): - scriptrelpath = os.path.dirname(relpath) + esdk_tools_path = os.path.dirname(relpath) relpath = os.path.join('layers', path, 'meta') if os.path.exists(os.path.join(baseoutpath, relpath, 'lib', 'oe')): core_meta_subdir = relpath d.setVar('oe_init_build_env_path', conf_initpath) - d.setVar('scriptrelpath', scriptrelpath) + d.setVar('esdk_tools_path', esdk_tools_path) # Write out config file for devtool import configparser @@ -627,14 +627,6 @@ def get_sdk_required_utilities(buildtools_fn, d): return ' '.join(sanity_required_utilities) install_tools() { - install -d ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk} - scripts="devtool recipetool oe-find-native-sysroot runqemu* wic" - for script in $scripts; do - for scriptfn in `find ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath} -maxdepth 1 -executable -name "$script"`; do - targetscriptfn="${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/$(basename $scriptfn)" - test -e ${targetscriptfn} || ln -rs ${scriptfn} ${targetscriptfn} - done - done touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase # find latest buildtools-tarball and install it @@ -713,7 +705,7 @@ sdk_ext_postinst() { # A bit of another hack, but we need this in the path only for devtool # so put it at the end of $PATH. - echo "export PATH=\"$target_sdk_dir/sysroots/${SDK_SYS}${bindir_nativesdk}:\$PATH\"" >> $env_setup_script + echo "export PATH=\"$target_sdk_dir/${esdk_tools_path}:\$PATH\"" >> $env_setup_script echo "printf 'SDK environment now set up; additionally you may now run devtool to perform development tasks.\nRun devtool --help for further details.\n'" >> $env_setup_script diff --git a/scripts/esdk-tools/devtool b/scripts/esdk-tools/devtool new file mode 120000 index 00000000000..176a01ca682 --- /dev/null +++ b/scripts/esdk-tools/devtool @@ -0,0 +1 @@ +../devtool \ No newline at end of file diff --git a/scripts/esdk-tools/oe-find-native-sysroot b/scripts/esdk-tools/oe-find-native-sysroot new file mode 120000 index 00000000000..d3493f3310a --- /dev/null +++ b/scripts/esdk-tools/oe-find-native-sysroot @@ -0,0 +1 @@ +../oe-find-native-sysroot \ No newline at end of file diff --git a/scripts/esdk-tools/recipetool b/scripts/esdk-tools/recipetool new file mode 120000 index 00000000000..60a95dd9360 --- /dev/null +++ b/scripts/esdk-tools/recipetool @@ -0,0 +1 @@ +../recipetool \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu b/scripts/esdk-tools/runqemu new file mode 120000 index 00000000000..ae7e7ad7c22 --- /dev/null +++ b/scripts/esdk-tools/runqemu @@ -0,0 +1 @@ +../runqemu \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu-addptable2image b/scripts/esdk-tools/runqemu-addptable2image new file mode 120000 index 00000000000..afcd00e79d9 --- /dev/null +++ b/scripts/esdk-tools/runqemu-addptable2image @@ -0,0 +1 @@ +../runqemu-addptable2image \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu-export-rootfs b/scripts/esdk-tools/runqemu-export-rootfs new file mode 120000 index 00000000000..a26fcf6110f --- /dev/null +++ b/scripts/esdk-tools/runqemu-export-rootfs @@ -0,0 +1 @@ +../runqemu-export-rootfs \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu-extract-sdk b/scripts/esdk-tools/runqemu-extract-sdk new file mode 120000 index 00000000000..cc858aaad55 --- /dev/null +++ b/scripts/esdk-tools/runqemu-extract-sdk @@ -0,0 +1 @@ +../runqemu-extract-sdk \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu-gen-tapdevs b/scripts/esdk-tools/runqemu-gen-tapdevs new file mode 120000 index 00000000000..dbdf79134cf --- /dev/null +++ b/scripts/esdk-tools/runqemu-gen-tapdevs @@ -0,0 +1 @@ +../runqemu-gen-tapdevs \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu-ifdown b/scripts/esdk-tools/runqemu-ifdown new file mode 120000 index 00000000000..0097693ca3d --- /dev/null +++ b/scripts/esdk-tools/runqemu-ifdown @@ -0,0 +1 @@ +../runqemu-ifdown \ No newline at end of file diff --git a/scripts/esdk-tools/runqemu-ifup b/scripts/esdk-tools/runqemu-ifup new file mode 120000 index 00000000000..41026d2c0a4 --- /dev/null +++ b/scripts/esdk-tools/runqemu-ifup @@ -0,0 +1 @@ +../runqemu-ifup \ No newline at end of file diff --git a/scripts/esdk-tools/wic b/scripts/esdk-tools/wic new file mode 120000 index 00000000000..a9d908aa254 --- /dev/null +++ b/scripts/esdk-tools/wic @@ -0,0 +1 @@ +../wic \ No newline at end of file From patchwork Fri Nov 3 10:28:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Kanavin X-Patchwork-Id: 33544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9B65C41535 for ; Fri, 3 Nov 2023 10:28:14 +0000 (UTC) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mx.groups.io with SMTP id smtpd.web11.48043.1699007293787472331 for ; Fri, 03 Nov 2023 03:28:14 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=MQTgj5qo; spf=pass (domain: gmail.com, ip: 209.85.128.43, mailfrom: alex.kanavin@gmail.com) Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-4083740f92dso13931175e9.3 for ; Fri, 03 Nov 2023 03:28:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699007292; x=1699612092; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tXqcaGcgaharJaj8Nq56+ev5XHwg1SS04RwWBLI/ZkU=; b=MQTgj5qoRps2/ki1qMha6w0ql1hzUjeibzBf8F1eXHVZCvQ1E4UBveBV5+mvtegMjc CMV8D6bSCk9lxY8ba4Hbdakw2X4Nm7ThPUDjzvULp+wgNlBdEtSejtMNKDG/nE9uTIVn VDfOzIQgWtMylamHq4eKTTo89ZrU3YcUvCQgd07JvlNckJd01z20YULJ2b/20yDiBZhv s5w7HQqzRq50jT04dOwwE1up+T7+kYMXmPEks75dci+s/1GoT2Z0Iv2QNPYtCrhOv5oI b1pk1yU2wPS0vA0F+8TBNkLH/mcCSW+fRD5U2DvJHtHO6rV6/qi9KE61YhUuvxwtpI+W +MFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699007292; x=1699612092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tXqcaGcgaharJaj8Nq56+ev5XHwg1SS04RwWBLI/ZkU=; b=gFURdHa5zy4eu4YfyOwMDmE5WnePHnFZFSGe0j2jbSgD+3pEq/dTrxAsl23rTBq32m mwGCswcgkl0fr8Qs+w/hDfb1HmyDM2bMQ+mgJbTyl30AXrPRYUymsrYfSsZ14+T4ifPy mCUQtHJumg1DEfqYrw9/lEM6VlIWNKI7K/qUnfQPkPBN4CRpCMvPC2xun4xl6aLAPosD 6n/ea+Jn2vbq/up4cgYTmM4+kes31y35F7piCA0UcsBkvd+XIdgpX3q+8vgt7SCzWh5G ffH4nop1vz+NWEU0WicjDu3/jmlsJJttwPVs8nX2wBmVUXxz7G+42AnEbK4YOvba16Ue AG2w== X-Gm-Message-State: AOJu0YyxnKruiB48fZUH689ED6BoBVIpa+O0nU+hExFdo72AuKFfoSxG /swfncfd1wTMQyb40z6CNyhVBX3/klc= X-Google-Smtp-Source: AGHT+IGK/JLeZI4CJ2L7o4iBMZEsnsydb3iRpc1nFjfyv1wyYuO1+HmYiTAfZThcR5DJURRHUHXQAw== X-Received: by 2002:a05:600c:190c:b0:402:bcac:5773 with SMTP id j12-20020a05600c190c00b00402bcac5773mr18155875wmq.38.1699007291949; Fri, 03 Nov 2023 03:28:11 -0700 (PDT) Received: from Zen2.lab.linutronix.de. (drugstore.linutronix.de. [80.153.143.164]) by smtp.gmail.com with ESMTPSA id y14-20020a05600c2b0e00b00405718cbeadsm1976643wme.1.2023.11.03.03.28.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Nov 2023 03:28:11 -0700 (PDT) From: Alexander Kanavin X-Google-Original-From: Alexander Kanavin To: openembedded-core@lists.openembedded.org Cc: Alexander Kanavin Subject: [PATCH 4/4] populate_sdk_ext: split copy_buildsystem() into logical steps defined as functions Date: Fri, 3 Nov 2023 11:28:06 +0100 Message-Id: <20231103102806.2332746-4-alex@linutronix.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20231103102806.2332746-1-alex@linutronix.de> References: <20231103102806.2332746-1-alex@linutronix.de> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 03 Nov 2023 10:28:14 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/190151 copy_buildsystem() has become far too large and needs to be split into shorter and more understandable pieces; some of those pieces will be reused for the purpose of providing esdk mode directly from a plain yocto build. Signed-off-by: Alexander Kanavin --- meta/classes-recipe/populate_sdk_ext.bbclass | 59 ++++++++++++++++---- 1 file changed, 47 insertions(+), 12 deletions(-) diff --git a/meta/classes-recipe/populate_sdk_ext.bbclass b/meta/classes-recipe/populate_sdk_ext.bbclass index 53adc868d47..a20f38fb210 100644 --- a/meta/classes-recipe/populate_sdk_ext.bbclass +++ b/meta/classes-recipe/populate_sdk_ext.bbclass @@ -186,12 +186,7 @@ def create_filtered_tasklist(d, sdkbasepath, tasklistfile, conf_initpath): if os.path.exists(localconf + '.bak'): os.replace(localconf + '.bak', localconf) -python copy_buildsystem () { - import re - import shutil - import glob - import oe.copy_buildsystem - +def copy_bitbake_and_layers(d, baseoutpath, derivative): oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT') conf_bbpath = '' @@ -200,13 +195,7 @@ python copy_buildsystem () { # Copy in all metadata layers + bitbake (as repositories) buildsystem = oe.copy_buildsystem.BuildSystem('extensible SDK', d) - baseoutpath = d.getVar('SDK_OUTPUT') + '/' + d.getVar('SDKPATH') - #check if custome templateconf path is set - use_custom_templateconf = d.getVar('SDK_CUSTOM_TEMPLATECONF') - - # Determine if we're building a derivative extensible SDK (from devtool build-sdk) - derivative = (d.getVar('SDK_DERIVATIVE') or '') == '1' if derivative: workspace_name = 'orig-workspace' else: @@ -231,6 +220,9 @@ python copy_buildsystem () { d.setVar('oe_init_build_env_path', conf_initpath) d.setVar('esdk_tools_path', esdk_tools_path) + return (conf_initpath, conf_bbpath, core_meta_subdir, sdkbblayers) + +def write_devtool_config(d, baseoutpath, conf_bbpath, conf_initpath, core_meta_subdir): # Write out config file for devtool import configparser config = configparser.ConfigParser() @@ -247,10 +239,12 @@ python copy_buildsystem () { with open(os.path.join(baseoutpath, 'conf', 'devtool.conf'), 'w') as f: config.write(f) +def write_unlocked_sigs(d, baseoutpath): unlockedsigs = os.path.join(baseoutpath, 'conf', 'unlocked-sigs.inc') with open(unlockedsigs, 'w') as f: pass +def write_bblayers_conf(d, baseoutpath, sdkbblayers): # Create a layer for new recipes / appends bbpath = d.getVar('BBPATH') env = os.environ.copy() @@ -279,6 +273,9 @@ python copy_buildsystem () { f.write(' $' + '{SDKBASEMETAPATH}/workspace \\\n') f.write(' "\n') +def copy_uninative(d, baseoutpath): + import shutil + # Copy uninative tarball # For now this is where uninative.bbclass expects the tarball if bb.data.inherits_class('uninative', d): @@ -288,6 +285,12 @@ python copy_buildsystem () { bb.utils.mkdirhier(uninative_outdir) shutil.copy(uninative_file, uninative_outdir) + return uninative_checksum + +def write_local_conf(d, baseoutpath, derivative, core_meta_subdir, uninative_checksum): + #check if custome templateconf path is set + use_custom_templateconf = d.getVar('SDK_CUSTOM_TEMPLATECONF') + env_passthrough = (d.getVar('BB_ENV_PASSTHROUGH_ADDITIONS') or '').split() env_passthrough_values = {} @@ -457,6 +460,9 @@ python copy_buildsystem () { f.write(line) f.write('\n') +def prepare_locked_cache(d, baseoutpath, conf_initpath): + import shutil + # Filter the locked signatures file to just the sstate tasks we are interested in excluded_targets = get_sdk_install_targets(d, images_only=True) sigfile = d.getVar('WORKDIR') + '/locked-sigs.inc' @@ -560,6 +566,9 @@ python copy_buildsystem () { f = os.path.join(root, name) os.remove(f) +def write_manifest(d, baseoutpath): + import glob + # Write manifest file # Note: at the moment we cannot include the env setup script here to keep # it updated, since it gets modified during SDK installation (see @@ -583,6 +592,32 @@ python copy_buildsystem () { continue chksum = bb.utils.sha256_file(fn) f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath))) + + +python copy_buildsystem () { + import oe.copy_buildsystem + + baseoutpath = d.getVar('SDK_OUTPUT') + '/' + d.getVar('SDKPATH') + + # Determine if we're building a derivative extensible SDK (from devtool build-sdk) + derivative = (d.getVar('SDK_DERIVATIVE') or '') == '1' + + conf_initpath, conf_bbpath, core_meta_subdir, sdkbblayers = copy_bitbake_and_layers(d, baseoutpath, derivative) + + write_devtool_config(d, baseoutpath, conf_bbpath, conf_initpath, core_meta_subdir) + + write_unlocked_sigs(d, baseoutpath) + + write_bblayers_conf(d, baseoutpath, sdkbblayers) + + uninative_checksum = copy_uninative(d, baseoutpath) + + write_local_conf(d, baseoutpath, derivative, core_meta_subdir, uninative_checksum) + + prepare_locked_cache(d, baseoutpath, conf_initpath) + + write_manifest(d, baseoutpath) + } def get_current_buildtools(d):