From patchwork Sun Aug 6 21:13:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charles-Antoine Couret X-Patchwork-Id: 28487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACB3CC04E69 for ; Sun, 6 Aug 2023 21:14:08 +0000 (UTC) Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by mx.groups.io with SMTP id smtpd.web11.19654.1691356441176008502 for ; Sun, 06 Aug 2023 14:14:01 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@mind.be header.s=google header.b=eor8JAN+; spf=pass (domain: essensium.com, ip: 209.85.221.49, mailfrom: charles-antoine.couret@essensium.com) Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-3175f17a7baso2827277f8f.0 for ; Sun, 06 Aug 2023 14:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mind.be; s=google; t=1691356439; x=1691961239; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LYhqziA1aSTdq1Ymgxyf/OWY+6CjckhuuYlfvfxE87U=; b=eor8JAN+H+kloxKA+V7RpKQX6KXCGap9Hd2GVzPaUid+vdzSXxPAJhY3lM1P44OWaG uVYu+SVuWCYgj4RQxBoethrAgQpJ5UhrAmIHLQH3GOl3fNCQ6E7XHLfZGS8uA88et+6C JjpfQ73gzzHNXMBtp0zsJwwNBNah6Sv96eBH/6AC6dWoZqaKCW02Wy5S+dGxw0h7vH/u /mZigQ8WCxnReW6XNOpUQomN6BnemIqEBYFKPwDX87XDQ2YAk30YuF+W3htoBtt2fn6w QKqpU+4SNQudgI4xYD2IKvbMbVozHthht+MKKqPCLwkDxDLSGg/CyKSCglwC2Y0uFQRw Qurg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691356439; x=1691961239; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LYhqziA1aSTdq1Ymgxyf/OWY+6CjckhuuYlfvfxE87U=; b=R8u5dAOObPhzA+8BVG/jP6HIlt7lkrrVZBBL1e2x3IotoU79jy1wICaf4lUiFLxGxh 67Bzh8zVS2diZEFqup7ZHHvn5jDeGXt/3CQrF/QNyrH2IO7UWgKCSBz4s+fHXk/DFgpl kfzU+WPLxWBs2lSyARCh6vtJfE7Qukx7x+z5/cTqKUk+T7mzW/r/gHyiseHGWKYsU9z8 NoPfdTEA7kPxHmD+sTZeI9/dz6x3epMiGfnwLAv7t9PK3owWgbGKXbj0kxOeenPmyR2I a2cuwphj/A4MSbYiiYXe2vZwrUbbF1I5weAzu2nn/STiR4wtG89Xhuk9k9Mbau3ycS3t FCMA== X-Gm-Message-State: AOJu0YzPEbljYk4Eq3B6/r9YiSg0XjoABwCDP+jPjUU9eQZtOteT8wgG 8UTz0axuFKpiN94Eo/c/bT8TrCg/fEZwbzQrORA= X-Google-Smtp-Source: AGHT+IGYgOronmvBDpPYYX9eHDkwmP5X6XXZvmH2YEhVwAh33NDXHNtmlUS8OEHEkONE0CZ/k/LP0w== X-Received: by 2002:a5d:54d2:0:b0:317:e025:5a5c with SMTP id x18-20020a5d54d2000000b00317e0255a5cmr1681322wrv.48.1691356439672; Sun, 06 Aug 2023 14:13:59 -0700 (PDT) Received: from Bishop.fritz.box ([2a02:578:85c6:1101:e7a2:3f2c:a83f:5e92]) by smtp.gmail.com with ESMTPSA id 21-20020a05600c021500b003fe1a092925sm8612055wmi.19.2023.08.06.14.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 14:13:59 -0700 (PDT) From: Charles-Antoine Couret To: openembedded-core@lists.openembedded.org Cc: Charles-Antoine Couret Subject: [PATCH 1/5 v2] image_types: add python function to get the IMAGE_FILE_MAXSIZE:fstype value Date: Sun, 6 Aug 2023 23:13:42 +0200 Message-ID: <20230806211348.1191553-2-charles-antoine.couret@mind.be> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230806211348.1191553-1-charles-antoine.couret@mind.be> References: <20230806211348.1191553-1-charles-antoine.couret@mind.be> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 06 Aug 2023 21:14:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/185575 It returns 0 if the variable is not set for this filesystem. In case of fixed partitionning where the rootfs partition can't exceed an amount of bytes, there is currently no automatic and no generic way to have this requirement met in any case. Until now, ROOTFS_SIZE value got from directory_size() does not takes into account the size of required metadata for the filesystem itself (and does not work well for other block size than 4k BTW). Obviously it's a difficult task which depends on rootfs size and filesystem type. The workaround was to set IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE to add the required extra margins. But when the final rootfs is closed to the maximum size, it's difficult to adjust them correctly And if you remove or add new recipes in your image, you've to recompute these margins to have enough space for these metadata when the rootfs is small, and to not have too big final file when the rootfs is big. It's cumbersome and error prone to just have a build failure when the final output can't be flashed into the partition. The solution is to follow how it's implemented in buildroot by having a specific variable, here IMAGE_FILE_MAXSIZE, to create the final sparse file and trying to fill it with the content of rootfs. If there is enough space, margins are well compressed and does not consume space in the filesystem. If there is no enough space, an error is triggered to warm the developer before trying to use it in the device. If IMAGE_FILE_MAXSIZE is not set, the idea is to keep the previous behaviour for compatibility reason and to met other requirements. Signed-off-by: Charles-Antoine Couret --- documentation/ref-manual/variables.rst | 14 ++++++++++++++ meta/classes-recipe/image_types.bbclass | 7 +++++++ 2 files changed, 21 insertions(+) diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index fc29e476cd..d0e476d2eb 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -3476,6 +3476,20 @@ system and gives an overview of their function and contents. variable, see the ":ref:`dev-manual/customizing-images:customizing images using custom \`\`image_features\`\` and \`\`extra_image_features\`\``" section in the Yocto Project Development Tasks Manual. + :term:`IMAGE_FILE_MAXSIZE` + Specifies the maximum size in kilobytes to create the image file for a + specific image type, which corresponds to the value set in + :term:`IMAGE_FSTYPES`, (e.g. ``ext3``, + ``btrfs``, and so forth). When setting this variable, you should use + an override for the associated type. Here is an example:: + + IMAGE_FILE_MAXSIZE:ext4 = "8192" + + It overrides the :term:`IMAGE_OVERHEAD_FACTOR` and + :term:`IMAGE_ROOTFS_EXTRA_SPACE` mechanism for some filesystems. + If the maximum size is below the required size to store the rootfs content, + the operation will fail. + :term:`IMAGE_FSTYPES` Specifies the formats the OpenEmbedded build system uses during the build when creating the root filesystem. For example, setting diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass index 023eb87537..33c65e8282 100644 --- a/meta/classes-recipe/image_types.bbclass +++ b/meta/classes-recipe/image_types.bbclass @@ -54,6 +54,13 @@ def imagetypes_getdepends(d): # Sort the set so that ordering is consistant return " ".join(sorted(deps)) +def get_max_image_size(d, fs): + max_size = d.getVar("IMAGE_FILE_MAXSIZE:%s" % fs) + if max_size is not None: + return max_size + + return 0 + XZ_COMPRESSION_LEVEL ?= "-9" XZ_INTEGRITY_CHECK ?= "crc32" From patchwork Sun Aug 6 21:13:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charles-Antoine Couret X-Patchwork-Id: 28489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD998C3DA40 for ; Sun, 6 Aug 2023 21:14:08 +0000 (UTC) Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by mx.groups.io with SMTP id smtpd.web11.19656.1691356442447119628 for ; Sun, 06 Aug 2023 14:14:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@mind.be header.s=google header.b=cPDzJxkl; spf=pass (domain: essensium.com, ip: 209.85.128.45, mailfrom: charles-antoine.couret@essensium.com) Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-3fe1e1142caso38952515e9.0 for ; Sun, 06 Aug 2023 14:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mind.be; s=google; t=1691356441; x=1691961241; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6An0wH99JHOGFPer94PqRj8JhNlTy2QEcU9rLas5jno=; b=cPDzJxkl4HRcbvsAwInX8qR5Uy8I6lMmDdY676OSpuOAYYCKWf47XJa3sz3rDSGMWO +FvrPINyO8KjLwV+xgbkK0oFd7qqAMcD/5VuSe+GAWOClrtfpFD0jN8BG1jSCu3WoDiW DVH9j6rnw4u+j4C8E56Zbzxbr81e9EN/ZuDNwS+szTeP8iV9k8iOv5ycCPqeVSszXP7p FALKpeGPZCSoYcp2AwqG+pH//rpSGyGjg5mxciomkNt9OxTrxFqsbu6NR6HUQoWEiJUU J3XdAClKsKgKRgVz8eZ5hjBy15DHaIOezqJed7Pxe/SpyLfF/ycJ58XqwJSRJP00tUGO IOlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691356441; x=1691961241; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6An0wH99JHOGFPer94PqRj8JhNlTy2QEcU9rLas5jno=; b=Mj8yS0Ppu5X/xD4jHxyZrcPbiMMhtpLSHrk+IYRyzAhDp8/WrmJFCCNSoss6P26+Bd v2b2n4KOxKWZjEpZVyffx2nQL+Hwc7mV9xyMT45Cn6cUJyMidKRDWQBlgqF48/tsgSYg ClE9CGcdOigLbIBVXSWeYIOuQrOAOJfRouQTKFZE82OndavTKJjL3zRtDcgy1rhHHUyN BWSi/qQBBP6EdIEb70Ku3Otv75EqhXTKbC7O7NE1X4WmXfSKu91BrRh3s5f4KALA4KFq 5q5y9AUAcWfS1KJuQ+KSPMOO4wHr8mKWGocmEZpsWLD+bVevZtktKH1l6Z4HEV7a98xS E37w== X-Gm-Message-State: AOJu0Yzu2uMru1XEdwDEs3womJdnR98svGBh3xLuCZmt6MO01QMPsmJx TDHvEH/UQvbENhL63tvi7L50XeJBdASTdRE9d9Q= X-Google-Smtp-Source: AGHT+IGVwGWh+Smfq6vZbWdfimvxxsEd1PsOVByTaIPHXOyO42sLxdQYsit9ToGIQKbS4yOTeMh8gw== X-Received: by 2002:a7b:c453:0:b0:3fc:d5:dc14 with SMTP id l19-20020a7bc453000000b003fc00d5dc14mr5536654wmi.5.1691356441035; Sun, 06 Aug 2023 14:14:01 -0700 (PDT) Received: from Bishop.fritz.box ([2a02:578:85c6:1101:e7a2:3f2c:a83f:5e92]) by smtp.gmail.com with ESMTPSA id 21-20020a05600c021500b003fe1a092925sm8612055wmi.19.2023.08.06.14.14.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 14:14:00 -0700 (PDT) From: Charles-Antoine Couret To: openembedded-core@lists.openembedded.org Cc: Charles-Antoine Couret Subject: [PATCH 2/5 v2] image_types: use IMAGE_FILE_MAXSIZE variable for ext2/3/4 image types Date: Sun, 6 Aug 2023 23:13:43 +0200 Message-ID: <20230806211348.1191553-3-charles-antoine.couret@mind.be> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230806211348.1191553-1-charles-antoine.couret@mind.be> References: <20230806211348.1191553-1-charles-antoine.couret@mind.be> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 06 Aug 2023 21:14:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/185576 If defined, this variable value overrides the size of ext* partition file created by mkfs. Otherwise previous logic based on ROOTFS_SIZE variable is used. It should be set when the final file size would not be above a specific value due to fixed partitionning for example. Signed-off-by: Charles-Antoine Couret --- meta/classes-recipe/image_types.bbclass | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass index 33c65e8282..2ec41c6e54 100644 --- a/meta/classes-recipe/image_types.bbclass +++ b/meta/classes-recipe/image_types.bbclass @@ -79,24 +79,32 @@ IMAGE_CMD:cramfs = "mkfs.cramfs ${IMAGE_ROOTFS} ${IMGDEPLOYDIR}/${IMAGE_NAME}${I oe_mkext234fs () { fstype=$1 + image_file_maxsize=$2 extra_imagecmd="" + rootfs_file_size=$ROOTFS_SIZE - if [ $# -gt 1 ]; then - shift + if [ $# -gt 2 ]; then + shift 2 extra_imagecmd=$@ fi + + if [[ "${image_file_maxsize}" -ne 0 ]]; then + rootfs_file_size=${image_file_maxsize} + fi + # If generating an empty image the size of the sparse block should be large # enough to allocate an ext4 filesystem using 4096 bytes per inode, this is # about 60K, so dd needs a minimum count of 60, with bs=1024 (bytes per IO) eval local COUNT=\"0\" eval local MIN_COUNT=\"60\" - if [ $ROOTFS_SIZE -lt $MIN_COUNT ]; then + if [ $rootfs_file_size -lt $MIN_COUNT ]; then eval COUNT=\"$MIN_COUNT\" fi + # Create a sparse image block - bbdebug 1 Executing "dd if=/dev/zero of=${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype seek=$ROOTFS_SIZE count=$COUNT bs=1024" - dd if=/dev/zero of=${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype seek=$ROOTFS_SIZE count=$COUNT bs=1024 + bbdebug 1 Executing "dd if=/dev/zero of=${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype seek=$rootfs_file_size count=$COUNT bs=1024" + dd if=/dev/zero of=${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype seek=$rootfs_file_size count=$COUNT bs=1024 bbdebug 1 "Actual Rootfs size: `du -s ${IMAGE_ROOTFS}`" bbdebug 1 "Actual Partition size: `stat -c '%s' ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype`" bbdebug 1 Executing "mkfs.$fstype -F $extra_imagecmd ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype -d ${IMAGE_ROOTFS}" @@ -105,9 +113,9 @@ oe_mkext234fs () { fsck.$fstype -pvfD ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.$fstype || [ $? -le 3 ] } -IMAGE_CMD:ext2 = "oe_mkext234fs ext2 ${EXTRA_IMAGECMD}" -IMAGE_CMD:ext3 = "oe_mkext234fs ext3 ${EXTRA_IMAGECMD}" -IMAGE_CMD:ext4 = "oe_mkext234fs ext4 ${EXTRA_IMAGECMD}" +IMAGE_CMD:ext2 = "oe_mkext234fs ext2 ${@get_max_image_size(d, 'ext2')} ${EXTRA_IMAGECMD}" +IMAGE_CMD:ext3 = "oe_mkext234fs ext3 ${@get_max_image_size(d, 'ext3')} ${EXTRA_IMAGECMD}" +IMAGE_CMD:ext4 = "oe_mkext234fs ext4 ${@get_max_image_size(d, 'ext4')} ${EXTRA_IMAGECMD}" MIN_BTRFS_SIZE ?= "16384" IMAGE_CMD:btrfs () { From patchwork Sun Aug 6 21:13:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charles-Antoine Couret X-Patchwork-Id: 28486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AACA7C04A6A for ; Sun, 6 Aug 2023 21:14:08 +0000 (UTC) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by mx.groups.io with SMTP id smtpd.web10.19680.1691356443767383686 for ; Sun, 06 Aug 2023 14:14:04 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@mind.be header.s=google header.b=PQhjtMaC; spf=pass (domain: essensium.com, ip: 209.85.128.43, mailfrom: charles-antoine.couret@essensium.com) Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-3fe426b86a8so30959215e9.3 for ; Sun, 06 Aug 2023 14:14:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mind.be; s=google; t=1691356442; x=1691961242; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9ZG1iJZoCpSpl4eL30M5IH8WQPHYpPpyh/WJOq0ap5k=; b=PQhjtMaClWlsN+AuirBvmYFvxiUQxImkT9F96WzYWfAVyR8MEHjHZVXhsf3MS77vxZ INeHpcuGWLCJV4iuFj0AcXiObBqyueC3rDqId5NaQBIfkekMQm7icIilC9UgT1s57E6w NMhtnksmv13mQ7FLDVFYPbOdoBfqR/lFlRPshg1VU2gGfgYZH1oyU/FT0V4qEmGiVaGg iiJPjCAxXkUvoeRu7GIABAdPHtXp5GIgTcEGMHPDbggHAU9ySeSwD+HXO6Z+O84H5Md+ +6Ir6DdSRjBpf2R6Wo51JdhRKAxy9xnNRBXcMQxTmzyHT97wqZ3YsOY1BU285XVot3lS O+Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691356442; x=1691961242; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9ZG1iJZoCpSpl4eL30M5IH8WQPHYpPpyh/WJOq0ap5k=; b=HWrs7xSDlR1FKRpMQrPOUyMuwECOdxGbhen1S9/wKsum+Dq4IpxjFL12mmKppmkw60 EUv9/PDR73dhSahS3f06AYBlZG06k3F88YQ7d08WLozb/Bk7reSihCeZBxov29kkgqP+ u3KNpq1YQGj+Szzj8g8oFMy8QIf/cXqk9TFnRhaEZW+6YP1kOVIIj386CblGGkfXwET4 rr1RqdrRm/BixI0QZjgrHJIhV9el7T/d8uWBXoEaqaIYXdb0V5c9fPMJEt3q7uJ7EQvr Cksmby3ZkhCG+HMF/NMaJkBNVhSkP4wfg9Ghrx+D7ADLxA2bQseg2WbGm3q/Su30k6E/ Z2TA== X-Gm-Message-State: AOJu0YxYzFBov9nY+smtBMrkw1heJFHOj/CvTgnjvHRp2+UbdqOwhFWk uMABYVGiYjG9YTsyd/qhsp7j0+s3EdincXkNYmk= X-Google-Smtp-Source: AGHT+IEUlZp0z0yZxgme9SNNBSmNAhTaHjXgu+8fX+cnwkqAZ171PZ+TY21KbKIWyuE1sGJDY0VoRA== X-Received: by 2002:a05:600c:2106:b0:3fb:e206:ca5f with SMTP id u6-20020a05600c210600b003fbe206ca5fmr5497597wml.31.1691356442319; Sun, 06 Aug 2023 14:14:02 -0700 (PDT) Received: from Bishop.fritz.box ([2a02:578:85c6:1101:e7a2:3f2c:a83f:5e92]) by smtp.gmail.com with ESMTPSA id 21-20020a05600c021500b003fe1a092925sm8612055wmi.19.2023.08.06.14.14.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 14:14:01 -0700 (PDT) From: Charles-Antoine Couret To: openembedded-core@lists.openembedded.org Cc: Charles-Antoine Couret Subject: [PATCH 3/5 v2] image_types: use IMAGE_FILE_MAXSIZE variable for btrfs image types Date: Sun, 6 Aug 2023 23:13:44 +0200 Message-ID: <20230806211348.1191553-4-charles-antoine.couret@mind.be> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230806211348.1191553-1-charles-antoine.couret@mind.be> References: <20230806211348.1191553-1-charles-antoine.couret@mind.be> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 06 Aug 2023 21:14:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/185577 If defined, this variable value overrides the size of btrfs partition file created by mkfs. Otherwise previous logic based on ROOTFS_SIZE variable is used. It should be set when the final file size would not be above a specific value due to fixed partitionning for example. Signed-off-by: Charles-Antoine Couret --- meta/classes-recipe/image_types.bbclass | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass index 2ec41c6e54..6e54f2f13a 100644 --- a/meta/classes-recipe/image_types.bbclass +++ b/meta/classes-recipe/image_types.bbclass @@ -120,6 +120,12 @@ IMAGE_CMD:ext4 = "oe_mkext234fs ext4 ${@get_max_image_size(d, 'ext4')} ${EXTRA_I MIN_BTRFS_SIZE ?= "16384" IMAGE_CMD:btrfs () { size=${ROOTFS_SIZE} + image_file_maxsize=${@get_max_image_size(d, "btrfs")} + + if [[ ${image_file_maxsize} -ne 0 ]]; then + size=${image_file_maxsize} + fi + if [ ${size} -lt ${MIN_BTRFS_SIZE} ] ; then size=${MIN_BTRFS_SIZE} bbwarn "Rootfs size is too small for BTRFS. Filesystem will be extended to ${size}K" From patchwork Sun Aug 6 21:13:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charles-Antoine Couret X-Patchwork-Id: 28485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95996C41513 for ; Sun, 6 Aug 2023 21:14:08 +0000 (UTC) Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by mx.groups.io with SMTP id smtpd.web10.19682.1691356445023978036 for ; Sun, 06 Aug 2023 14:14:05 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@mind.be header.s=google header.b=aRXELDld; spf=pass (domain: essensium.com, ip: 209.85.128.54, mailfrom: charles-antoine.couret@essensium.com) Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-3fe5c0e58c0so3310495e9.3 for ; Sun, 06 Aug 2023 14:14:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mind.be; s=google; t=1691356443; x=1691961243; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RGkLhKjL3aU3rNFb1YmJ1b1W58upYs8XA4oGpDwpga4=; b=aRXELDldtzkZ/d44quvEwFW6CVtkP7EBZKcDKl4lK2rFEzClPtdF+QBaEQdvN5SL4g RVjCIlUQlc7oSaqSEoyr1j9IGVGX9wwp444//I56aaI8piwlyzasJw28FB5aUhVPCfRm iJm1AVl1ECXUHD8NVzD1RtUfQtWAIdcJkUxAsH5PEnMPGUCyR3+ogGTJwQOfJxr0qMm4 zwoZuufNgPHczokcsxzfQ68i3H7sqjqMyma6CtHtkws/lACKE1kaPpNnfvqSIT/LnGd2 +ezBKJfAQ6Wx/pu3H6k74HBj4/OpozastL1EBkBCjceaRuUi8dskKd2a3ezWb6p7Qt6c JyaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691356443; x=1691961243; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RGkLhKjL3aU3rNFb1YmJ1b1W58upYs8XA4oGpDwpga4=; b=HUMqTO9ns6r4TilansEOTnEJTwT4hyBuaZYkymFL7nl14J/8awRooxyhr6OYIsduQV CYJSadD+OPJ20hs+EG4UDPmLrccnkZo9sk5jb+Npsr7dMwqzKJHqqyNK1z7r75Oavyc0 RpV017GodU9ACs3OsIOfKWvVRaxLUr0OfSYHkzME0j4Q1qLgWkC/F8wB8AHhW5IhYMDH J0MiMW9KH+Z7wNhxNtJGcfY/brRW65t95QjTUq0xtYG9G784mn6JvoFhLPS8o3TWGn3g a5f+F5a+p7FyXFCHtrVV8tPX9cgfzP3qaeqZBbZEaWBELkcLYSE0a0vABM1EpX2Ys/9X FFpw== X-Gm-Message-State: AOJu0Yz0cr66W3QFM3hqpUmnDlOe8CWX2jRhZeyVs0F39lDWYbyiRNYL SKqlVXS6oKma69APIN2EKEXq25qiKXjc0fOa/Zg= X-Google-Smtp-Source: AGHT+IHD+i8WIc0eRZLok8FnQAys3Nzpu52L2fnmcl7bn13sXCMtxGoXArVJ3qewAU3H6OOTaHF2rA== X-Received: by 2002:a1c:7706:0:b0:3f8:fc96:6bfd with SMTP id t6-20020a1c7706000000b003f8fc966bfdmr6287960wmi.17.1691356443599; Sun, 06 Aug 2023 14:14:03 -0700 (PDT) Received: from Bishop.fritz.box ([2a02:578:85c6:1101:e7a2:3f2c:a83f:5e92]) by smtp.gmail.com with ESMTPSA id 21-20020a05600c021500b003fe1a092925sm8612055wmi.19.2023.08.06.14.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 14:14:03 -0700 (PDT) From: Charles-Antoine Couret To: openembedded-core@lists.openembedded.org Cc: Charles-Antoine Couret Subject: [PATCH 4/5 v2] image_types: use IMAGE_FILE_MAXSIZE variable for f2fs image types Date: Sun, 6 Aug 2023 23:13:45 +0200 Message-ID: <20230806211348.1191553-5-charles-antoine.couret@mind.be> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230806211348.1191553-1-charles-antoine.couret@mind.be> References: <20230806211348.1191553-1-charles-antoine.couret@mind.be> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 06 Aug 2023 21:14:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/185578 If defined, this variable value overrides the size of f2fs partition file created by mkfs. Otherwise previous logic based on ROOTFS_SIZE variable is used. It should be set when the final file size would not be above a specific value due to fixed partitionning for example. Signed-off-by: Charles-Antoine Couret --- meta/classes-recipe/image_types.bbclass | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass index 6e54f2f13a..51b2b6ea1d 100644 --- a/meta/classes-recipe/image_types.bbclass +++ b/meta/classes-recipe/image_types.bbclass @@ -263,6 +263,12 @@ IMAGE_CMD:f2fs () { # 500M the standard IMAGE_OVERHEAD_FACTOR does not work, so add additional # space here when under 500M size=${ROOTFS_SIZE} + image_file_maxsize=${@get_max_image_size(d, "f2fs")} + + if [[ "${image_file_maxsize}" -ne 0 ]]; then + size=${image_file_maxsize} + fi + if [ ${size} -lt ${MIN_F2FS_SIZE} ] ; then size=${MIN_F2FS_SIZE} bbwarn "Rootfs size is too small for F2FS. Filesystem will be extended to ${size}K" From patchwork Sun Aug 6 21:13:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charles-Antoine Couret X-Patchwork-Id: 28488 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93ECDC0015E for ; Sun, 6 Aug 2023 21:14:08 +0000 (UTC) Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) by mx.groups.io with SMTP id smtpd.web11.19657.1691356446459433016 for ; Sun, 06 Aug 2023 14:14:06 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@mind.be header.s=google header.b=DxTuZUVd; spf=pass (domain: essensium.com, ip: 209.85.128.47, mailfrom: charles-antoine.couret@essensium.com) Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-3fe4cdb727cso20758675e9.0 for ; Sun, 06 Aug 2023 14:14:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mind.be; s=google; t=1691356445; x=1691961245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IeDx4xZ10/rJV5piXYBSCWSCabdbEoFEH60J0TbGrR4=; b=DxTuZUVdg43C/+u5oCYT0lZd+gDomVyjwnOl+ELfScEbpHi67MOLoo4XCH4fHkMZhq 3AMv9BNG8JbpAUTlecYyjmR7RpgzQwG6YGhjQidOHWvzEQLb+JFYpsKs6mTSdNbDaoyq qPpO8GBaoEmJ/RhEu6FuaBUtk4w0T/N2M3mqh4uID35eScg92giv+HZ+olnirRd0Q2Jd LensgdQvd3Xvv/ocLlMYT4+7eBpBqSPLISSlm3dapChlm3EVeRHDXuMsRjUV8SyC+61v Vg7xuNcjohA95dtdKDBFxQKZA6roqL3mBMmApR0lNtmbMOHzbxoQkhTgjMOMN3B5y1iu lZ7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691356445; x=1691961245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IeDx4xZ10/rJV5piXYBSCWSCabdbEoFEH60J0TbGrR4=; b=C9sTqe/9x83AoBq78zKvFITkomw54yjRUCqcasn9vFZ18mo4/F+juTzjVPF4AlHvUf v2AgLu8Vq7fa0s/YQlJElmgQfJML8Wgccw+cZFTjnF43iEC+9pdqHEpZxlXVrGcfKNrV QCNAx4WUVf4i/pJVi0Ob3BwvlVqwuWrNbBT7mkmuJwjH2WqoqqVe9xjre8FS/xg+kw2F JFXT5lNekDnHqHyLKrkKt69E5c+a9zGkRVfHrw4o007aArbL4m+yU9at7iJ+9VUH1cGt ewfU9i/BMuAgvCdwAuf+2Z2YzxFSbRJVu7xcBg5O3O8gqQyx+ihCbgDNWlioEhSro2iS bqUg== X-Gm-Message-State: AOJu0YwkGmjHAaHyJgUBbO9pL873J7kCdQ4ts62XVaoIT/vZrmuKwsmV IM3biX/e36fcGN+V/e4MXU3ImDoJWxF+v+lYYYI= X-Google-Smtp-Source: AGHT+IGX6+1R06v69V0yL5nsnlF0qtNJyvu/S1gUs7iSKJRhJ2oPJvoq+B2GtrZrqbD39Kmarkbz5g== X-Received: by 2002:a7b:c5d2:0:b0:3fe:2011:a7bf with SMTP id n18-20020a7bc5d2000000b003fe2011a7bfmr5775270wmk.6.1691356445042; Sun, 06 Aug 2023 14:14:05 -0700 (PDT) Received: from Bishop.fritz.box ([2a02:578:85c6:1101:e7a2:3f2c:a83f:5e92]) by smtp.gmail.com with ESMTPSA id 21-20020a05600c021500b003fe1a092925sm8612055wmi.19.2023.08.06.14.14.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 06 Aug 2023 14:14:04 -0700 (PDT) From: Charles-Antoine Couret To: openembedded-core@lists.openembedded.org Cc: Charles-Antoine Couret Subject: [PATCH 5/5 v2] image: add check_image_max_size as post function to check file size against IMAGE_FILE_MAXSIZE Date: Sun, 6 Aug 2023 23:13:46 +0200 Message-ID: <20230806211348.1191553-6-charles-antoine.couret@mind.be> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230806211348.1191553-1-charles-antoine.couret@mind.be> References: <20230806211348.1191553-1-charles-antoine.couret@mind.be> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sun, 06 Aug 2023 21:14:08 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/185579 Trigger an error if the final size is above IMAGE_FILE_MAXSIZE value. Which is relevant if the partition size is fixed and the user wants to be sure that the image can be entirely installed into its partition. If the variable is not set, no error is trigger. It works for all filesystems. Signed-off-by: Charles-Antoine Couret --- meta/classes-recipe/image.bbclass | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass index e0dfba4a42..bf47f3dea3 100644 --- a/meta/classes-recipe/image.bbclass +++ b/meta/classes-recipe/image.bbclass @@ -509,6 +509,7 @@ python () { d.appendVarFlag(task, 'prefuncs', ' ' + debug + ' set_image_size') d.prependVarFlag(task, 'postfuncs', 'create_symlinks ') + d.prependVarFlag(task, 'postfuncs', 'check_image_max_size ') d.appendVarFlag(task, 'subimages', ' ' + ' '.join(subimages)) d.appendVarFlag(task, 'vardeps', ' ' + ' '.join(vardeps)) d.appendVarFlag(task, 'vardepsexclude', ' DATETIME DATE ' + ' '.join(vardepsexclude)) @@ -610,6 +611,35 @@ python create_symlinks() { bb.note("Skipping symlink, source does not exist: %s -> %s" % (dst, src)) } +# +# Check if image size is lighter than maximum size +# +python check_image_max_size() { + def get_max_image_size(d, fs): + max_size = d.getVar("IMAGE_FILE_MAXSIZE:%s" % fs) + if max_size is not None: + return int(max_size) + + return None + + deploy_dir = d.getVar('IMGDEPLOYDIR') + img_name = d.getVar('IMAGE_NAME') + taskname = d.getVar("BB_CURRENTTASK") + subimages = (d.getVarFlag("do_" + taskname, 'subimages', False) or "").split() + imgsuffix = d.getVarFlag("do_" + taskname, 'imgsuffix') or d.expand("${IMAGE_NAME_SUFFIX}.") + + for type in subimages: + file_name = os.path.join(deploy_dir, img_name + imgsuffix + type) + + if os.path.exists(file_name): + file_size = os.stat(file_name).st_size / 1024 + max_size = get_max_image_size(d, type) + if max_size is not None: + if file_size > max_size: + bb.error("The image %s size %d(K) exceeds IMAGE_FILE_MAXSIZE: %d(K)" % \ + (file_name, file_size, max_size)) +} + MULTILIBRE_ALLOW_REP =. "${base_bindir}|${base_sbindir}|${bindir}|${sbindir}|${libexecdir}|${sysconfdir}|${nonarch_base_libdir}/udev|/lib/modules/[^/]*/modules.*|" MULTILIB_CHECK_FILE = "${WORKDIR}/multilib_check.py" MULTILIB_TEMP_ROOTFS = "${WORKDIR}/multilib"