[v2,2/2] image_types.bbclass: add support for sparse images

Android-targeted devices support so called sparse image format. This is
the raw filesystem image with dropped zero blocks. This way the image
takes less space and the bootloaders and/or filesystem manipulation
tools can flash it quicker (as they do not have to write useless
sequences of zeroes).

Add new image conversion type called 'sparse'to be able to generate
filesystem images in the sparse format.

Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
---
 meta/classes-recipe/image_types.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

We perhaps need tests as well?

Alex

On Tue, 14 Nov 2023 at 22:04, Dmitry Baryshkov <dbaryshkov@gmail.com> wrote:
>
> Android-targeted devices support so called sparse image format. This is
> the raw filesystem image with dropped zero blocks. This way the image
> takes less space and the bootloaders and/or filesystem manipulation
> tools can flash it quicker (as they do not have to write useless
> sequences of zeroes).
>
> Add new image conversion type called 'sparse'to be able to generate
> filesystem images in the sparse format.
>
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> ---
>  meta/classes-recipe/image_types.bbclass | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass
> index d615b41ed1ed..d0c9514b36a3 100644
> --- a/meta/classes-recipe/image_types.bbclass
> +++ b/meta/classes-recipe/image_types.bbclass
> @@ -328,7 +328,7 @@ IMAGE_TYPES:append:x86-64 = " hddimg iso"
>  # CONVERSION_CMD/DEPENDS.
>  COMPRESSIONTYPES ?= ""
>
> -CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync zsync ${COMPRESSIONTYPES}"
> +CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync zsync sparse ${COMPRESSIONTYPES}"
>  CONVERSION_CMD:lzma = "lzma -k -f -7 ${IMAGE_NAME}.${type}"
>  CONVERSION_CMD:gz = "gzip -f -9 -n -c --rsyncable ${IMAGE_NAME}.${type} > ${IMAGE_NAME}.${type}.gz"
>  CONVERSION_CMD:bz2 = "pbzip2 -f -k ${IMAGE_NAME}.${type}"
> @@ -355,6 +355,7 @@ CONVERSION_CMD:qcow2 = "qemu-img convert -O qcow2 ${IMAGE_NAME}.${type} ${IMAGE_
>  CONVERSION_CMD:base64 = "base64 ${IMAGE_NAME}.${type} > ${IMAGE_NAME}.${type}.base64"
>  CONVERSION_CMD:zsync = "zsyncmake_curl ${IMAGE_NAME}.${type}"
>  CONVERSION_CMD:gzsync = "zsyncmake_curl -z ${IMAGE_NAME}.${type}"
> +CONVERSION_CMD:sparse = "img2simg -c ${IMAGE_NAME}.${type} ${IMAGE_NAME}.${type}.sparse"
>  CONVERSION_DEPENDS_lzma = "xz-native"
>  CONVERSION_DEPENDS_gz = "pigz-native"
>  CONVERSION_DEPENDS_bz2 = "pbzip2-native"
> @@ -375,6 +376,7 @@ CONVERSION_DEPENDS_vhdx = "qemu-system-native"
>  CONVERSION_DEPENDS_vhd = "qemu-system-native"
>  CONVERSION_DEPENDS_zsync = "zsync-curl-native"
>  CONVERSION_DEPENDS_gzsync = "zsync-curl-native"
> +CONVERSION_DEPENDS_sparse = "rust-android-sparse-native"
>
>  RUNNABLE_IMAGE_TYPES ?= "ext2 ext3 ext4"
>  RUNNABLE_MACHINE_PATTERNS ?= "qemu"
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#190521): https://lists.openembedded.org/g/openembedded-core/message/190521
> Mute This Topic: https://lists.openembedded.org/mt/102592509/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

On Wed, 2023-11-15 at 00:04 +0300, Dmitry Baryshkov wrote:
> Android-targeted devices support so called sparse image format. This is
> the raw filesystem image with dropped zero blocks. This way the image
> takes less space and the bootloaders and/or filesystem manipulation
> tools can flash it quicker (as they do not have to write useless
> sequences of zeroes).
> 
> Add new image conversion type called 'sparse'to be able to generate
> filesystem images in the sparse format.
> 
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@linaro.org>
> ---
>  meta/classes-recipe/image_types.bbclass | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass
> index d615b41ed1ed..d0c9514b36a3 100644
> --- a/meta/classes-recipe/image_types.bbclass
> +++ b/meta/classes-recipe/image_types.bbclass
> @@ -328,7 +328,7 @@ IMAGE_TYPES:append:x86-64 = " hddimg iso"
>  # CONVERSION_CMD/DEPENDS.
>  COMPRESSIONTYPES ?= ""
>  
> -CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync zsync ${COMPRESSIONTYPES}"
> +CONVERSIONTYPES = "gz bz2 lzma xz lz4 lzo zip 7zip zst sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap u-boot vmdk vhd vhdx vdi qcow2 base64 gzsync zsync sparse ${COMPRESSIONTYPES}"
>  CONVERSION_CMD:lzma = "lzma -k -f -7 ${IMAGE_NAME}.${type}"
>  CONVERSION_CMD:gz = "gzip -f -9 -n -c --rsyncable ${IMAGE_NAME}.${type} > ${IMAGE_NAME}.${type}.gz"
>  CONVERSION_CMD:bz2 = "pbzip2 -f -k ${IMAGE_NAME}.${type}"

I think we may need to be more specific about what kind of "sparse"
image this is, maybe "android-sparse" ?

Cheers,

Richard

On 14 Nov 2023, at 21:04, Dmitry Baryshkov via lists.openembedded.org <dbaryshkov=gmail.com@lists.openembedded.org> wrote:
> 
> Android-targeted devices support so called sparse image format. This is
> the raw filesystem image with dropped zero blocks. This way the image
> takes less space and the bootloaders and/or filesystem manipulation
> tools can flash it quicker (as they do not have to write useless
> sequences of zeroes).
> 
> Add new image conversion type called 'sparse'to be able to generate
> filesystem images in the sparse format.

These files are not traditional sparse files but a new format which empty blocks identified in them, right? Then this name is very confusing and as RP says, maybe android-sparse is better.

It’s a shame to have two identical tools, bmap-tools and this...

> +CONVERSION_DEPENDS_sparse = "rust-android-sparse-native”

As conversion commands can be in any class, you could add a new class to whatever layer has this recipe instead of adding directly to core.

Ross