From patchwork Mon Dec 13 21:14:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Bergin X-Patchwork-Id: 1445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6132C433F5 for ; Mon, 13 Dec 2021 21:14:57 +0000 (UTC) Received: from mailrelay3-2.pub.mailoutpod1-cph3.one.com (mailrelay3-2.pub.mailoutpod1-cph3.one.com [46.30.212.2]) by mx.groups.io with SMTP id smtpd.web09.17342.1639430095427550347 for ; Mon, 13 Dec 2021 13:14:56 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@berginkonsult.se header.s=rsa1 header.b=oNojhXqK; spf=none, err=permanent DNS error (domain: berginkonsult.se, ip: 46.30.212.2, mailfrom: peter@berginkonsult.se) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=berginkonsult.se; s=rsa1; h=content-transfer-encoding:mime-version:references:in-reply-to:message-id:date: subject:cc:to:from:from; bh=XFBJqIdYIvRFMDkhLWPgq1aeEGWd+bHZFKJFpDIBLtY=; b=oNojhXqKzlagQZEeX4ApcGonbMRheXgIclRH5IjImCwq30DxyAf0vlbYNuEIWyD4tPEF85YKqkUDi Jzkdq+38x5ZQbeXsDllKQPkfXVNWN7KyNVsl6fZPSfebaOsSD74KntkYnx50Lnqf/WVgv9fPWXzlv+ BFFJ9HLUZN8L5jnHAYZMIb3KTtQuud6lAlJNAQNitxvhWa/QaZoxRutdofuF5nGFm69cH1OR/WpaT/ trXAduHoj1XoF+cvtje4hYlWjNlArLHxjfrX1aKNWwoKSanW5m43Qtfy97MP06Lu53UaRi0DRlwL/+ zHRNRqSDTeMP1npBAY4r2qNhYIZ+jfQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=berginkonsult.se; s=ed1; h=content-transfer-encoding:mime-version:references:in-reply-to:message-id:date: subject:cc:to:from:from; bh=XFBJqIdYIvRFMDkhLWPgq1aeEGWd+bHZFKJFpDIBLtY=; b=lP3q/nY0pNucT3YHEAZTjAe1nkNMFcbtjDYfpOK4Lg5XuXJv57Ys5Jtk6KgSVhYjy/5ugBot6TO8k Hq8J3z+Bg== X-HalOne-Cookie: 231381319bfbdf5ac5ddc3463a267df738e46070 X-HalOne-ID: b6b4fbf6-5c59-11ec-9170-d0431ea8bb03 Received: from pbergin-7420.. (ua-213-113-157-96.bbcust.telenor.se [213.113.157.96]) by mailrelay3.pub.mailoutpod1-cph3.one.com (Halon) with ESMTPSA id b6b4fbf6-5c59-11ec-9170-d0431ea8bb03; Mon, 13 Dec 2021 21:14:52 +0000 (UTC) From: Peter Bergin To: openembedded-core@lists.openembedded.org Cc: Kristian Amlie , Peter Bergin Subject: [RFC PATCH] do_image: Implement IMAGE_ROOTFS_EXCLUDE_PATH feature. Date: Mon, 13 Dec 2021 22:14:16 +0100 Message-Id: <20211213211416.46449-2-peter@berginkonsult.se> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211213211416.46449-1-peter@berginkonsult.se> References: <20211213211416.46449-1-peter@berginkonsult.se> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 13 Dec 2021 21:14:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/159659 From: Kristian Amlie This is a direct followup from the earlier 6602392db3d39 commit in wic. It works more or less the same way: The variable specifies a list of directories relative to the root of the rootfs, and these directories will be excluded from the resulting rootfs image. If an entry ends with a slash, only the contents are omitted, not the directory itself. Since the intended use of the variable is to exclude certain directories from the rootfs, and then include said directories in other partitions, it is not natural for this variable to be respected for image creators that create multi partition images. These can turn the feature off locally by defining: do_image_myfs[respect_exclude_path] = "0" Specifically, "wic" and "multiubi" come with the feature disabled. Signed-off-by: Kristian Amlie Signed-off-by: Peter Bergin --- meta/classes/image.bbclass | 84 +++++++++++++++++++++++++++- meta/classes/image_types.bbclass | 1 + meta/classes/image_types_wic.bbclass | 1 + 3 files changed, 84 insertions(+), 2 deletions(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 2b0ce4a988..1c2ddd274e 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -125,7 +125,8 @@ def rootfs_variables(d): 'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS', 'IMAGE_LINGUAS_COMPLEMENTARY', 'IMAGE_LOCALES_ARCHIVE', 'MULTILIBRE_ALLOW_REP','MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS', 'PACKAGE_ARCHS','PACKAGE_CLASSES','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','USE_DEVFS', - 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 'REPRODUCIBLE_TIMESTAMP_ROOTFS', 'IMAGE_INSTALL_DEBUGFS'] + 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 'REPRODUCIBLE_TIMESTAMP_ROOTFS', 'IMAGE_INSTALL_DEBUGFS', + 'IMAGE_ROOTFS_EXCLUDE_PATH'] variables.extend(rootfs_command_variables(d)) variables.extend(variable_depends(d)) return " ".join(variables) @@ -497,8 +498,9 @@ python () { d.setVarFlag(task, 'func', '1') d.setVarFlag(task, 'fakeroot', '1') - d.appendVarFlag(task, 'prefuncs', ' ' + debug + ' set_image_size') + d.appendVarFlag(task, 'prefuncs', ' ' + debug + ' set_image_size prepare_excluded_directories') d.prependVarFlag(task, 'postfuncs', 'create_symlinks ') + d.appendVarFlag(task, 'postfuncs', ' cleanup_excluded_directories') d.appendVarFlag(task, 'subimages', ' ' + ' '.join(subimages)) d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps)) d.appendVarFlag(task, 'vardepsexclude', ' DATETIME DATE ' + ' '.join(vardepsexclude)) @@ -507,6 +509,84 @@ python () { bb.build.addtask(task, 'do_image_complete', after, d) } +python prepare_excluded_directories() { + exclude_var = d.getVar('IMAGE_ROOTFS_EXCLUDE_PATH') + if not exclude_var: + return + + taskname = d.getVar("BB_CURRENTTASK") + + if d.getVarFlag('do_%s' % taskname, 'respect_exclude_path') == '0': + bb.debug(1, "'IMAGE_ROOTFS_EXCLUDE_PATH' is set but 'respect_exclude_path' variable flag is 0 for this image type, so ignoring it") + return + + import shutil + from oe.path import copyhardlinktree + + exclude_list = exclude_var.split() + + rootfs_orig = d.getVar('IMAGE_ROOTFS') + # We need a new rootfs directory we can delete files from. Copy to + # workdir. + new_rootfs = os.path.realpath(oe.path.join(d.getVar("WORKDIR"), "rootfs.%s" % taskname)) + + if os.path.lexists(new_rootfs): + shutil.rmtree(new_rootfs) + + copyhardlinktree(rootfs_orig, new_rootfs) + + for orig_path in exclude_list: + path = orig_path + if os.path.isabs(path): + bb.fatal("IMAGE_ROOTFS_EXCLUDE_PATH: Must be relative: %s" % orig_path) + + full_path = os.path.realpath(oe.path.join(new_rootfs, path)) + + # Disallow climbing outside of parent directory using '..', + # because doing so could be quite disastrous (we will delete the + # directory). + if not full_path.startswith(new_rootfs): + bb.fatal("'%s' points to a path outside the rootfs" % orig_path) + + if path.endswith(os.sep): + # Delete content only. + for entry in os.listdir(full_path): + full_entry = oe.path.join(full_path, entry) + if os.path.isdir(full_entry) and not os.path.islink(full_entry): + shutil.rmtree(full_entry) + else: + os.remove(full_entry) + else: + # Delete whole directory. + shutil.rmtree(full_path) + + # Save old value for cleanup later. + d.setVar('IMAGE_ROOTFS_ORIG', rootfs_orig) + d.setVar('IMAGE_ROOTFS', new_rootfs) +} + +python cleanup_excluded_directories() { + exclude_var = d.getVar('IMAGE_ROOTFS_EXCLUDE_PATH') + if not exclude_var: + return + + taskname = d.getVar("BB_CURRENTTASK") + + if d.getVarFlag('do_%s' % taskname, 'respect_exclude_path') == '0': + return + + import shutil + + rootfs_dirs_excluded = d.getVar('IMAGE_ROOTFS') + rootfs_orig = d.getVar('IMAGE_ROOTFS_ORIG') + # This should never happen, since we should have set it to a different + # directory in the prepare function. + assert rootfs_dirs_excluded != rootfs_orig + + shutil.rmtree(rootfs_dirs_excluded) + d.setVar('IMAGE_ROOTFS', rootfs_orig) +} + # # Compute the rootfs size # diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass index f643ed3ce7..59abc827f7 100644 --- a/meta/classes/image_types.bbclass +++ b/meta/classes/image_types.bbclass @@ -196,6 +196,7 @@ IMAGE_CMD:multiubi () { multiubi_mkfs "${mkubifs_args}" "${ubinize_args}" "${name}" done } +do_image_multiubi[respect_exclude_path] = "0" IMAGE_CMD:ubi () { multiubi_mkfs "${MKUBIFS_ARGS}" "${UBINIZE_ARGS}" diff --git a/meta/classes/image_types_wic.bbclass b/meta/classes/image_types_wic.bbclass index e3863c88a9..47855c3175 100644 --- a/meta/classes/image_types_wic.bbclass +++ b/meta/classes/image_types_wic.bbclass @@ -69,6 +69,7 @@ IMAGE_CMD:wic () { } IMAGE_CMD:wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES TOPDIR" do_image_wic[cleandirs] = "${WORKDIR}/build-wic" +do_image_wic[respect_exclude_path] = "0" PSEUDO_IGNORE_PATHS .= ",${WORKDIR}/build-wic"