[4/6] reproducible_build: clamp the rootfs tarball mtime to the epoch

Submitted by Ross Burton on Dec. 1, 2020, 3:23 p.m. | Patch ID: 178781

Details

Message ID 20201201152305.2375723-4-ross.burton@arm.com
State New
Headers show

Commit Message

Ross Burton Dec. 1, 2020, 3:23 p.m. 
To handle files being written to the rootfs outside of the package manager,
clamp the mtimes to the epoch.

Signed-off-by: Ross Burton <ross.burton@arm.com>

---
 meta/classes/reproducible_build.bbclass | 3 +++
 1 file changed, 3 insertions(+)

-- 
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145125): https://lists.openembedded.org/g/openembedded-core/message/145125
Mute This Topic: https://lists.openembedded.org/mt/78635762/1003190
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [mhalstead@linuxfoundation.org]
-=-=-=-=-=-=-=-=-=-=-=-

Patch hide | download patch | download mbox 

diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index 2f3bd90b073..28a25fb492d 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -116,3 +116,6 @@  python () {
     if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
         d.appendVarFlag("do_unpack", "postfuncs", " create_source_date_epoch_stamp")
 }
+
+# Clamp the mtimes of the tarball image filesystem type
+IMAGE_CMD_tar_append = " --clamp-mtime --mtime=@${SOURCE_DATE_EPOCH}"

Comments

Richard Purdie Dec. 1, 2020, 3:45 p.m. 
On Tue, 2020-12-01 at 15:23 +0000, Ross Burton wrote:
> To handle files being written to the rootfs outside of the package

> manager,

> clamp the mtimes to the epoch.

> 

> Signed-off-by: Ross Burton <ross.burton@arm.com>

> ---

>  meta/classes/reproducible_build.bbclass | 3 +++

>  1 file changed, 3 insertions(+)

> 

> diff --git a/meta/classes/reproducible_build.bbclass

> b/meta/classes/reproducible_build.bbclass

> index 2f3bd90b073..28a25fb492d 100644

> --- a/meta/classes/reproducible_build.bbclass

> +++ b/meta/classes/reproducible_build.bbclass

> @@ -116,3 +116,6 @@ python () {

>      if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':

>          d.appendVarFlag("do_unpack", "postfuncs", "

> create_source_date_epoch_stamp")

>  }

> +

> +# Clamp the mtimes of the tarball image filesystem type

> +IMAGE_CMD_tar_append = " --clamp-mtime --

> mtime=@${SOURCE_DATE_EPOCH}"

> 


I'm not convinced this is correct as it could mess up the timestamps of
stamps set in other recipes? 

It's also tar specific and should really be across all image types?

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145130): https://lists.openembedded.org/g/openembedded-core/message/145130
Mute This Topic: https://lists.openembedded.org/mt/78635762/1003190
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [mhalstead@linuxfoundation.org]
-=-=-=-=-=-=-=-=-=-=-=-
Jate S Dec. 1, 2020, 11:25 p.m. 
I have started using the following code to set the mtimes of files written
after the build timestamp. It assumes that SOURCE_DATE_EPOCH of packages
are never in the future.

# Set to empty to allow custom function below to run
REPRODUCIBLE_TIMESTAMP_ROOTFS ?= ""

# Use DATETIME of build as SOURCE_DATE_EPOCH for image
export SOURCE_DATE_EPOCH = "${@int( time.mktime( time.strptime('${DATETIME}',
'%Y%m%d%H%M%S') ) ) }"

reproducible_final_image_task () {
if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
# Use commit time of tuxzilla root in usual Syntux build
REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}/../" log -1 --pretty=%ct
2>/dev/null`
if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(d.getVar("BBPATH"),
"conf/bitbake.conf")}`
fi
fi
# Set mtime of all files to a reproducible value
bbnote "reproducible_final_image_task: mtime set to
$REPRODUCIBLE_TIMESTAMP_ROOTFS"
# Files with mtime after build stamp should be part of rootfs creation
DAYS_SINCE_BUILD=`python3 -c "import time; print((time.time() -
${SOURCE_DATE_EPOCH}) / 86400)"`
find ${IMAGE_ROOTFS} -mtime -$DAYS_SINCE_BUILD -exec touch -h --date=@
$REPRODUCIBLE_TIMESTAMP_ROOTFS {} \;
fi
}


On Tue, Dec 1, 2020 at 12:16 PM Ross Burton <ross@burtonini.com> wrote:

> On Tue, 1 Dec 2020 at 15:45, Richard Purdie

> <richard.purdie@linuxfoundation.org> wrote:

> > I'm not convinced this is correct as it could mess up the timestamps of

> > stamps set in other recipes?

> >

> > It's also tar specific and should really be across all image types?

>

> Yes, it most likely should be done at rootfs time instead.

>

> Ross

>

> 

>

>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145163): https://lists.openembedded.org/g/openembedded-core/message/145163
Mute This Topic: https://lists.openembedded.org/mt/78635762/1003190
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [mhalstead@linuxfoundation.org]
-=-=-=-=-=-=-=-=-=-=-=-