diff mbox series

image_types.bbclass: Use xz default compression preset level

Message ID 20231030131253.1707354-1-niko.mauno@vaisala.com
State Accepted, archived
Commit 9ca62e24a6a0f5d2778b2b587646df7447e3c65f
Headers show
Series image_types.bbclass: Use xz default compression preset level | expand

Commit Message

Niko Mauno Oct. 30, 2023, 1:12 p.m. UTC 
From: Niko Mauno <niko.mauno@vaisala.com>

Commit ef0654f1453ff0afe98d7e921626b2a96cf2f6f6
("Set XZ_COMPRESSION_LEVEL to -9") changed the xz compression preset
level from previous value of -3 to -9. The commit message explains that
the change was made in order to be consistent with other compressors
that also use their best compression. However looking at xz man page,
under the compression preset level selection chapter there is mentioned
that

  The  differences  between  the  presets  are more significant than with gzip(1) and
  bzip2(1).  The selected compression settings determine the memory  requirements  of
  the  decompressor,  thus  using  a  too  high preset level might make it painful to
  decompress the file on an old system with little RAM.   Specifically,  it's  not  a
  good  idea  to  blindly  use  -9  for  everything like it often is with gzip(1) and
  bzip2(1).

which is then followed by a table, which mentions that the decompressor
memory requirement for preset -9 is 65 MiB, whereas for xz default
preset -6 it is just 9 MiB. Given that the use case where a device
running a Yocto generated Linux OS decompresses an ext4 root filesystem
image to non-volatile memory as part of firmware upgrade process is not
far-fetched, and considering that a range of these devices can run low
on available RAM when there are other applications running at the same
time, the lower decompressor memory requirement of the default preset
level makes sense in order to prevent an OOM situation from occurring.

This change was tested on a 32 CPU core build host with 128 GB RAM by
issuing

  $ bitbake -c cleansstate core-image-minimal core-image-sato
  $ time bitbake core-image-minimal
  $ time bitbake core-image-sato

With MACHINE="qemux86-64" and IMAGE_FSTYPES="ext4 ext4.xz" using
XZ_COMPRESSION_LEVEL values "-6" and "-9". In both cases the resulting
'ext4' image size remained same, 38141952 bytes for core-image-minimal,
and 565043200 bytes for core-image-sato.

The observation was that with this change there is a small increase in
the resulting 'ext4.xz' file size, and a build speed improvement that
was significant for larger rootfs image.

core-image  XZ  real time    time delta	   ext4.xz size      size delta
-----------------------------------------------------------------------
minimal     -9  0m44.992s                      15932508
minimal     -6  0m42.445s        -5.66%        16243484         +1.95%
sato        -9  2m40.828s                      85080416
sato        -6  1m38.891s       -38.51%        87447456         +2.78%

Regarding decompression speed, issuing following command in qemux86-64
target OS

  $ time xz -dkc --memlimit=MEMLIMIT core-image-sato-qemux86-64.rootfs.ext4.xz > /dev/null

using the lowest accepted value for MEMLIMIT for each case (providing a
lower value caused xz to exit with 'Memory usage limit reached' error)
showed that decompression time saw a minuscule improvement with the -6
compression preset level:

XZ   MEMLIMIT   real time
-------------------------
-9        65M    0m43.83s
-6         9M    0m43.28s

(In the above tables, XZ refers to XZ_COMPRESSION_LEVEL value used when
images were generated with Yocto).

Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
---
 meta/classes-recipe/image_types.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff mbox series

Patch

diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass
index 4aed64e27f..d615b41ed1 100644
--- a/meta/classes-recipe/image_types.bbclass
+++ b/meta/classes-recipe/image_types.bbclass
@@ -54,7 +54,7 @@  def imagetypes_getdepends(d):
     # Sort the set so that ordering is consistant
     return " ".join(sorted(deps))
 
-XZ_COMPRESSION_LEVEL ?= "-9"
+XZ_COMPRESSION_LEVEL ?= "-6"
 XZ_INTEGRITY_CHECK ?= "crc32"
 
 ZIP_COMPRESSION_LEVEL ?= "-9"