| Message ID | 20221224014946.2644943-1-luis.pinto.martins@gmail.com |
|---|---|
| State | Accepted, archived |
| Commit | edcd9ad333bc4e504594e8af83e8cb7007d2e35c |
| Headers | show |
| Series | rm_work.bbclass: use HOSTTOOLS 'rm' binary exclusively | expand |
Hi, I see this was accepted for master, is it okay to port this to dunfell and kirkstone as well ? Best regards, Luís Martins On Sat, 24 Dec 2022 at 01:49, Luis Martins <luis.pinto.martins@gmail.com> wrote: > The do_rm_work() task is using the first available 'rm' binary > available in PATH to remove files and folders. > However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE > contents, the function can be using the 'rm' binary available > in RECIPE_SYSROOT_NATIVE, a folder that will get removed. > This causes a sporadic race-condition when trying to access the > 'rm' binary of a folder already deleted. > Solve this by exclusively using the HOSTTOOLS 'rm' binary, as > this folder will not get removed. > > Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com> > --- > meta/classes/rm_work.bbclass | 15 +++++++++++---- > 1 file changed, 11 insertions(+), 4 deletions(-) > > diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass > index 1f28bc7187..8b5fe1b808 100644 > --- a/meta/classes/rm_work.bbclass > +++ b/meta/classes/rm_work.bbclass > @@ -33,6 +33,13 @@ BB_SCHEDULER ?= "completion" > BB_TASK_IONICE_LEVEL:task-rm_work = "3.0" > > do_rm_work () { > + # Force using the HOSTTOOLS 'rm' - otherwise the SYSROOT_NATIVE 'rm' > can be selected depending on PATH > + # Avoids race-condition accessing 'rm' when deleting WORKDIR folders > at the end of this function > + RM_BIN="$(PATH=${HOSTTOOLS_DIR} command -v rm)" > + if [ -z "${RM_BIN}" ]; then > + bbfatal "Binary 'rm' not found in HOSTTOOLS_DIR, cannot remove > WORKDIR data." > + fi > + > # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe. > for p in ${RM_WORK_EXCLUDE}; do > if [ "$p" = "${PN}" ]; then > @@ -79,7 +86,7 @@ do_rm_work () { > # sstate version since otherwise we'd need to leave > 'plaindirs' around > # such as 'packages' and 'packages-split' and these can be > large. No end > # of chain tasks depend directly on do_package anymore. > - rm -f -- $i; > + "${RM_BIN}" -f -- $i; > ;; > *_setscene*) > # Skip stamps which are already setscene versions > @@ -96,7 +103,7 @@ do_rm_work () { > ;; > esac > done > - rm -f -- $i > + "${RM_BIN}" -f -- $i > esac > done > > @@ -106,9 +113,9 @@ do_rm_work () { > # Retain only logs and other files in temp, safely ignore > # failures of removing pseudo folers on NFS2/3 server. > if [ $dir = 'pseudo' ]; then > - rm -rf -- $dir 2> /dev/null || true > + "${RM_BIN}" -rf -- $dir 2> /dev/null || true > elif ! echo "$excludes" | grep -q -w "$dir"; then > - rm -rf -- $dir > + "${RM_BIN}" -rf -- $dir > fi > done > } > -- > 2.25.1 > >
diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass index 1f28bc7187..8b5fe1b808 100644 --- a/meta/classes/rm_work.bbclass +++ b/meta/classes/rm_work.bbclass @@ -33,6 +33,13 @@ BB_SCHEDULER ?= "completion" BB_TASK_IONICE_LEVEL:task-rm_work = "3.0" do_rm_work () { + # Force using the HOSTTOOLS 'rm' - otherwise the SYSROOT_NATIVE 'rm' can be selected depending on PATH + # Avoids race-condition accessing 'rm' when deleting WORKDIR folders at the end of this function + RM_BIN="$(PATH=${HOSTTOOLS_DIR} command -v rm)" + if [ -z "${RM_BIN}" ]; then + bbfatal "Binary 'rm' not found in HOSTTOOLS_DIR, cannot remove WORKDIR data." + fi + # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe. for p in ${RM_WORK_EXCLUDE}; do if [ "$p" = "${PN}" ]; then @@ -79,7 +86,7 @@ do_rm_work () { # sstate version since otherwise we'd need to leave 'plaindirs' around # such as 'packages' and 'packages-split' and these can be large. No end # of chain tasks depend directly on do_package anymore. - rm -f -- $i; + "${RM_BIN}" -f -- $i; ;; *_setscene*) # Skip stamps which are already setscene versions @@ -96,7 +103,7 @@ do_rm_work () { ;; esac done - rm -f -- $i + "${RM_BIN}" -f -- $i esac done @@ -106,9 +113,9 @@ do_rm_work () { # Retain only logs and other files in temp, safely ignore # failures of removing pseudo folers on NFS2/3 server. if [ $dir = 'pseudo' ]; then - rm -rf -- $dir 2> /dev/null || true + "${RM_BIN}" -rf -- $dir 2> /dev/null || true elif ! echo "$excludes" | grep -q -w "$dir"; then - rm -rf -- $dir + "${RM_BIN}" -rf -- $dir fi done }
The do_rm_work() task is using the first available 'rm' binary available in PATH to remove files and folders. However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE contents, the function can be using the 'rm' binary available in RECIPE_SYSROOT_NATIVE, a folder that will get removed. This causes a sporadic race-condition when trying to access the 'rm' binary of a folder already deleted. Solve this by exclusively using the HOSTTOOLS 'rm' binary, as this folder will not get removed. Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com> --- meta/classes/rm_work.bbclass | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)