Patchwork [1/1] archive.bbclass: archive work directory

login
register
mail settings
Submitter wenzong.fan@windriver.com
Date Jan. 6, 2012, 8:20 a.m.
Message ID <b63c8a5a0cf70a797fd49844b7854151a58f579e.1325837822.git.wenzong.fan@windriver.com>
Download mbox | patch
Permalink /patch/18593/
State New
Headers show

Comments

wenzong.fan@windriver.com - Jan. 6, 2012, 8:20 a.m.
From: Wenzong Fan <wenzong.fan@windriver.com>

Some legal departments believe a complete archive of the work directory
is required for certain license compliance issues. We could therefore
do with a class which archives up the work directories in each build and
provide them so those legal departments can be happy.

Implementations:

Add a new class named 'archive.bbclass' to provide task 'do_archive',
and get it called after 'do_patch' before 'do_configure'.

Following cases should be considered to the sources dirs:
1) The sources dir is under $WORKDIR:
Just archive sources and temp/run.* up.

2) The sources dir is outside of $WORKDIR, the only package is gcc:
Copy its sources and temp/run.* to a temporary dir and then archive
them up.

3) The sources dir is equal to $WORKDIR:
Just archive whole work dir up.

[YOCTO #1590]

Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com>
---
 meta/classes/archive.bbclass |   42 ++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 42 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/archive.bbclass
Saul Wold - Jan. 6, 2012, 8:46 a.m.
On 01/06/2012 12:20 AM, wenzong.fan@windriver.com wrote:
> From: Wenzong Fan<wenzong.fan@windriver.com>
>
> Some legal departments believe a complete archive of the work directory
> is required for certain license compliance issues. We could therefore
> do with a class which archives up the work directories in each build and
> provide them so those legal departments can be happy.
>
> Implementations:
>
> Add a new class named 'archive.bbclass' to provide task 'do_archive',
> and get it called after 'do_patch' before 'do_configure'.
>
> Following cases should be considered to the sources dirs:
> 1) The sources dir is under $WORKDIR:
> Just archive sources and temp/run.* up.
>
> 2) The sources dir is outside of $WORKDIR, the only package is gcc:
> Copy its sources and temp/run.* to a temporary dir and then archive
> them up.
>
> 3) The sources dir is equal to $WORKDIR:
> Just archive whole work dir up.
>
> [YOCTO #1590]
>
> Signed-off-by: Wenzong Fan<wenzong.fan@windriver.com>
> ---
>   meta/classes/archive.bbclass |   42 ++++++++++++++++++++++++++++++++++++++++++
>   1 files changed, 42 insertions(+), 0 deletions(-)
>   create mode 100644 meta/classes/archive.bbclass
>
> diff --git a/meta/classes/archive.bbclass b/meta/classes/archive.bbclass
> new file mode 100644
> index 0000000..75ac090
> --- /dev/null
> +++ b/meta/classes/archive.bbclass
> @@ -0,0 +1,42 @@
> +# Archive the patched sources and build scripts to assist in license
> +# compliance by the end user or legal departments.
> +
> +ARCHIVE_DIR = "${TMPDIR}/archives/${MULTIMACH_TARGET_SYS}/"
> +do_archive[dirs] = "${ARCHIVE_DIR}"
> +
I see you set ARCHIVE_DIR, but I don' see you move or copy any files to 
this directory, am I missing something?

> +archive_do_archive() {
> +    # In mostly scenarios the $S is under $WORKDIR and has a separate
> +    # dir for storing the sources; but gcc is a special case, its sources
> +    # had been moved to the shared location 'tmp/work-shared/'
> +    if [[ -d ${S}&&  ${S} != ${WORKDIR} ]]; then
> +        if [[ ${S} =~ "/work-shared/gcc" ]]; then
> +            # Create temporary sources directory for gcc
> +            mkdir -p ${PF}/temp
> +            cp -r ${S} ${PF}
> +            cp -r ${S}/../temp/* ${PF}/temp
> +            cp -r ${WORKDIR}/temp/* ${PF}/temp
> +            tarbase=`pwd`
> +        else
> +            tarbase=`dirname ${WORKDIR}`
> +        fi
> +
> +        sourcedir=`basename ${S}`
> +        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF}/$sourcedir \
> +            ${PF}/temp --exclude log.do_*
> +
Also, do you want to name the tarball PF or BP, which deals with 
removing any multilib naming issues if the build it multilib.

Note BP = ${BPN}-${PV} so it removes the ${PR} if that is important 
(which it might be).

> +        # Remove the temporary gcc sources directory
> +        if [[ ${S} =~ "/work-shared/gcc"&&  -d ${PF} ]]; then
> +            rm -rf ${PF}
> +        fi
> +    fi
> +
> +    # Just archive whole build directory up when $S is equal to $WORKDIR
> +    if [[ -d ${S}&&  ${S} == ${WORKDIR} ]]; then
> +        tarbase=`dirname ${WORKDIR}`
> +        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF} --exclude log.do_*
> +    fi
> +}
> +
> +addtask do_archive after do_patch before do_configure
> +
> +EXPORT_FUNCTIONS do_archive
Phil Blundell - Jan. 6, 2012, 5:39 p.m.
On Fri, 2012-01-06 at 16:20 +0800, wenzong.fan@windriver.com wrote:
> +    if [[ -d ${S} && ${S} != ${WORKDIR} ]]; then
> +        if [[ ${S} =~ "/work-shared/gcc" ]]; then
> +            # Create temporary sources directory for gcc
> +            mkdir -p ${PF}/temp
> +            cp -r ${S} ${PF}
> +            cp -r ${S}/../temp/* ${PF}/temp
> +            cp -r ${WORKDIR}/temp/* ${PF}/temp
> +            tarbase=`pwd`

Special-casing gcc like this seems a bit lame.  Is that check really
needed?

Also, will the above code work with all shells?  It looks a bit
bash-specific to me.

p.
wenzong.fan@windriver.com - Jan. 9, 2012, 2:20 a.m.
On 01/06/2012 04:46 PM, Saul Wold wrote:
> On 01/06/2012 12:20 AM, wenzong.fan@windriver.com wrote:
>> From: Wenzong Fan<wenzong.fan@windriver.com>
>>
>> Some legal departments believe a complete archive of the work directory
>> is required for certain license compliance issues. We could therefore
>> do with a class which archives up the work directories in each build and
>> provide them so those legal departments can be happy.
>>
>> Implementations:
>>
>> Add a new class named 'archive.bbclass' to provide task 'do_archive',
>> and get it called after 'do_patch' before 'do_configure'.
>>
>> Following cases should be considered to the sources dirs:
>> 1) The sources dir is under $WORKDIR:
>> Just archive sources and temp/run.* up.
>>
>> 2) The sources dir is outside of $WORKDIR, the only package is gcc:
>> Copy its sources and temp/run.* to a temporary dir and then archive
>> them up.
>>
>> 3) The sources dir is equal to $WORKDIR:
>> Just archive whole work dir up.
>>
>> [YOCTO #1590]
>>
>> Signed-off-by: Wenzong Fan<wenzong.fan@windriver.com>
>> ---
>>   meta/classes/archive.bbclass |   42 
>> ++++++++++++++++++++++++++++++++++++++++++
>>   1 files changed, 42 insertions(+), 0 deletions(-)
>>   create mode 100644 meta/classes/archive.bbclass
>>
>> diff --git a/meta/classes/archive.bbclass b/meta/classes/archive.bbclass
>> new file mode 100644
>> index 0000000..75ac090
>> --- /dev/null
>> +++ b/meta/classes/archive.bbclass
>> @@ -0,0 +1,42 @@
>> +# Archive the patched sources and build scripts to assist in license
>> +# compliance by the end user or legal departments.
>> +
>> +ARCHIVE_DIR = "${TMPDIR}/archives/${MULTIMACH_TARGET_SYS}/"
>> +do_archive[dirs] = "${ARCHIVE_DIR}"
>> +
> I see you set ARCHIVE_DIR, but I don' see you move or copy any files 
> to this directory, am I missing something?

It just gets bitbake creating '${ARCHIVE_DIR}' before the task started, 
and then 'do_archive' will be run in this dir.

>
>> +archive_do_archive() {
>> +    # In mostly scenarios the $S is under $WORKDIR and has a separate
>> +    # dir for storing the sources; but gcc is a special case, its 
>> sources
>> +    # had been moved to the shared location 'tmp/work-shared/'
>> +    if [[ -d ${S}&&  ${S} != ${WORKDIR} ]]; then
>> +        if [[ ${S} =~ "/work-shared/gcc" ]]; then
>> +            # Create temporary sources directory for gcc
>> +            mkdir -p ${PF}/temp
>> +            cp -r ${S} ${PF}
>> +            cp -r ${S}/../temp/* ${PF}/temp
>> +            cp -r ${WORKDIR}/temp/* ${PF}/temp
>> +            tarbase=`pwd`
>> +        else
>> +            tarbase=`dirname ${WORKDIR}`
>> +        fi
>> +
>> +        sourcedir=`basename ${S}`
>> +        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF}/$sourcedir \
>> +            ${PF}/temp --exclude log.do_*
>> +
> Also, do you want to name the tarball PF or BP, which deals with 
> removing any multilib naming issues if the build it multilib.
>
> Note BP = ${BPN}-${PV} so it removes the ${PR} if that is important 
> (which it might be).

Sorry I'm not clear your exact meanings here, do you mean the tarball 
name with '${PF}' will have any issues?

Thanks
Wenzong

>
>> +        # Remove the temporary gcc sources directory
>> +        if [[ ${S} =~ "/work-shared/gcc"&&  -d ${PF} ]]; then
>> +            rm -rf ${PF}
>> +        fi
>> +    fi
>> +
>> +    # Just archive whole build directory up when $S is equal to 
>> $WORKDIR
>> +    if [[ -d ${S}&&  ${S} == ${WORKDIR} ]]; then
>> +        tarbase=`dirname ${WORKDIR}`
>> +        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF} --exclude log.do_*
>> +    fi
>> +}
>> +
>> +addtask do_archive after do_patch before do_configure
>> +
>> +EXPORT_FUNCTIONS do_archive
>
wenzong.fan@windriver.com - Jan. 9, 2012, 3:31 a.m.
On 01/07/2012 01:39 AM, Phil Blundell wrote:
> On Fri, 2012-01-06 at 16:20 +0800, wenzong.fan@windriver.com wrote:
>    
>> +    if [[ -d ${S}&&  ${S} != ${WORKDIR} ]]; then
>> +        if [[ ${S} =~ "/work-shared/gcc" ]]; then
>> +            # Create temporary sources directory for gcc
>> +            mkdir -p ${PF}/temp
>> +            cp -r ${S} ${PF}
>> +            cp -r ${S}/../temp/* ${PF}/temp
>> +            cp -r ${WORKDIR}/temp/* ${PF}/temp
>> +            tarbase=`pwd`
>>      
> Special-casing gcc like this seems a bit lame.  Is that check really
> needed?
>    

Oh, we might have other packages under this dir, so I'll remove the 
special-casing 'gcc' here.
> Also, will the above code work with all shells?  It looks a bit
> bash-specific to me.
>    
I assume that the bitbake have to run with 'bash', looks like this is 
not right on all distributions.

I'll update this.

Thanks
Wenzong

> p.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>

Patch

diff --git a/meta/classes/archive.bbclass b/meta/classes/archive.bbclass
new file mode 100644
index 0000000..75ac090
--- /dev/null
+++ b/meta/classes/archive.bbclass
@@ -0,0 +1,42 @@ 
+# Archive the patched sources and build scripts to assist in license
+# compliance by the end user or legal departments.
+
+ARCHIVE_DIR = "${TMPDIR}/archives/${MULTIMACH_TARGET_SYS}/"
+do_archive[dirs] = "${ARCHIVE_DIR}"
+
+archive_do_archive() {
+    # In mostly scenarios the $S is under $WORKDIR and has a separate
+    # dir for storing the sources; but gcc is a special case, its sources
+    # had been moved to the shared location 'tmp/work-shared/'
+    if [[ -d ${S} && ${S} != ${WORKDIR} ]]; then
+        if [[ ${S} =~ "/work-shared/gcc" ]]; then
+            # Create temporary sources directory for gcc
+            mkdir -p ${PF}/temp
+            cp -r ${S} ${PF}
+            cp -r ${S}/../temp/* ${PF}/temp
+            cp -r ${WORKDIR}/temp/* ${PF}/temp
+            tarbase=`pwd`
+        else
+            tarbase=`dirname ${WORKDIR}`
+        fi
+
+        sourcedir=`basename ${S}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF}/$sourcedir \
+            ${PF}/temp --exclude log.do_*
+
+        # Remove the temporary gcc sources directory
+        if [[ ${S} =~ "/work-shared/gcc" && -d ${PF} ]]; then
+            rm -rf ${PF}
+        fi
+    fi
+
+    # Just archive whole build directory up when $S is equal to $WORKDIR
+    if [[ -d ${S} && ${S} == ${WORKDIR} ]]; then
+        tarbase=`dirname ${WORKDIR}`
+        tar -C $tarbase -cjf ${PF}.tar.bz2 ${PF} --exclude log.do_*
+    fi
+}
+
+addtask do_archive after do_patch before do_configure
+
+EXPORT_FUNCTIONS do_archive