[v2,2/2] buildhistory: support generating sha256 checksums of regular files

Submitted by Jacob Kroon on Jan. 9, 2019, 8:30 p.m. | Patch ID: 157664

Details

Message ID 20190109203051.12258-2-jacob.kroon@gmail.com
State New
Headers show

Commit Message

Jacob Kroon Jan. 9, 2019, 8:30 p.m.
Introduce 'sha256' in BUILDHISTORY_FEATURES and enable it by default
when doing reproducible builds.

When enabled this will additionally create:

  files-in-package-sha256.txt
  files-in-image-sha256.txt
  files-in-sdk-sha256.txt

containing the sha256 checksums of regular files.

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
---
 meta/classes/buildhistory.bbclass | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

Changes in v2:
 * Switch to sha256
 * Let find only build the arguments and execute sha256sum once
 * Use single quotes in inline python code

Patch hide | download patch | download mbox

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 33eb1b00f6..84f85da0bd 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -7,7 +7,8 @@ 
 # Copyright (C) 2007-2011 Koen Kooi <koen@openembedded.org>
 #
 
-BUILDHISTORY_FEATURES ?= "image package sdk"
+BUILDHISTORY_FEATURES ?= "image package sdk \
+  ${@ 'sha256' if bb.utils.to_boolean(d.getVar('BUILD_REPRODUCIBLE_BINARIES')) else ''}"
 BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
 BUILDHISTORY_DIR_IMAGE = "${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/${IMAGE_BASENAME}"
 BUILDHISTORY_DIR_PACKAGE = "${BUILDHISTORY_DIR}/packages/${MULTIMACH_TARGET_SYS}/${PN}"
@@ -526,7 +527,12 @@  buildhistory_list_files() {
 		eval ${FAKEROOTENV} ${FAKEROOTCMD} $find_cmd
 	else
 		eval $find_cmd
-	fi | sort -k5 | sed 's/ * -> $//' > $2 )
+	fi | sort -k5 | sed 's/ * -> $//' > $2
+	if [ "${@bb.utils.contains('BUILDHISTORY_FEATURES', 'sha256', '1', '0', d)}" = "1" ] ; then
+		sha256filename=$(echo $2 | sed 's/\.txt$/-sha256.txt/')
+		find -type f -exec sha256sum {} + | sort -k2 > $sha256filename
+		[ -s $sha256filename ] || rm $sha256filename # remove result if empty
+	fi )
 }
 
 buildhistory_list_pkg_files() {

Comments

Jacob Kroon Jan. 12, 2019, 7:35 p.m.
On Wed, Jan 9, 2019 at 9:31 PM Jacob Kroon <jacob.kroon@gmail.com> wrote:
>
> Introduce 'sha256' in BUILDHISTORY_FEATURES and enable it by default
> when doing reproducible builds.
>
> When enabled this will additionally create:
>
>   files-in-package-sha256.txt
>   files-in-image-sha256.txt
>   files-in-sdk-sha256.txt
>
> containing the sha256 checksums of regular files.
>
> Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
> ---
>  meta/classes/buildhistory.bbclass | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
>
> Changes in v2:
>  * Switch to sha256
>  * Let find only build the arguments and execute sha256sum once
>  * Use single quotes in inline python code
>
> diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> index 33eb1b00f6..84f85da0bd 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -7,7 +7,8 @@
>  # Copyright (C) 2007-2011 Koen Kooi <koen@openembedded.org>
>  #
>
> -BUILDHISTORY_FEATURES ?= "image package sdk"
> +BUILDHISTORY_FEATURES ?= "image package sdk \
> +  ${@ 'sha256' if bb.utils.to_boolean(d.getVar('BUILD_REPRODUCIBLE_BINARIES')) else ''}"
>  BUILDHISTORY_DIR ?= "${TOPDIR}/buildhistory"
>  BUILDHISTORY_DIR_IMAGE = "${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/${IMAGE_BASENAME}"
>  BUILDHISTORY_DIR_PACKAGE = "${BUILDHISTORY_DIR}/packages/${MULTIMACH_TARGET_SYS}/${PN}"
> @@ -526,7 +527,12 @@ buildhistory_list_files() {
>                 eval ${FAKEROOTENV} ${FAKEROOTCMD} $find_cmd
>         else
>                 eval $find_cmd
> -       fi | sort -k5 | sed 's/ * -> $//' > $2 )
> +       fi | sort -k5 | sed 's/ * -> $//' > $2
> +       if [ "${@bb.utils.contains('BUILDHISTORY_FEATURES', 'sha256', '1', '0', d)}" = "1" ] ; then
> +               sha256filename=$(echo $2 | sed 's/\.txt$/-sha256.txt/')
> +               find -type f -exec sha256sum {} + | sort -k2 > $sha256filename
> +               [ -s $sha256filename ] || rm $sha256filename # remove result if empty
> +       fi )
>  }
>
>  buildhistory_list_pkg_files() {
> --
> 2.11.0

Ignore this patch, I completely missed the "task"
BUILDHISTORY_FEATURE, which seems to do this already.