buildhistory.bbclass: add support for SRCREV logging

- create "latest_srcrev" for each recipe with the following format:
${BB_FILENAME},${SRC_URI},${SRCREV},${FROM_AUTOREV}

[YOCTO #3041]

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
---
 meta/classes/buildhistory.bbclass | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

On 02/27/2013 07:37 AM, Constantin Musca wrote:
> - create "latest_srcrev" for each recipe with the following format:
> ${BB_FILENAME},${SRC_URI},${SRCREV},${FROM_AUTOREV}
>
> [YOCTO #3041]
>
> Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
> ---
>   meta/classes/buildhistory.bbclass | 23 ++++++++++++++++++++++-
>   1 file changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
> index a20d03d..fa13ef5 100644
> --- a/meta/classes/buildhistory.bbclass
> +++ b/meta/classes/buildhistory.bbclass
> @@ -44,6 +44,11 @@ python buildhistory_emit_pkghistory() {
>               self.pr = "r0"
>               self.depends = ""
>               self.packages = ""
> +            self.bbfile = ""
> +            self.src_uri = ""
> +            self.srcrev = ""
> +            self.srcrev_autorev = ""
> +
>
>       class PackageInfo:
>           def __init__(self, name):
> @@ -151,6 +156,12 @@ python buildhistory_emit_pkghistory() {
>       pe = d.getVar('PE', True) or "0"
>       pv = d.getVar('PV', True)
>       pr = d.getVar('PR', True)
> +
> +    bbfile = d.getVar('BB_FILENAME', True)
> +    src_uri = d.getVar('SRC_URI', True)
> +    srcrev = d.getVar('SRCREV', True)
> +    srcrev_autorev = 'yes' if d.getVar('SRCREV', False) == 'AUTOINC' else 'no'
> +
>       packages = squashspaces(d.getVar('PACKAGES', True))
>
>       packagelist = packages.split()
> @@ -159,7 +170,7 @@ python buildhistory_emit_pkghistory() {
>       else:
>           # Remove files for packages that no longer exist
>           for item in os.listdir(pkghistdir):
> -            if item != "latest":
> +            if item != "latest" and item != "latest_srcrev":
>                   if item not in packagelist:
>                       subdir = os.path.join(pkghistdir, item)
>                       for subfile in os.listdir(subdir):
> @@ -171,6 +182,10 @@ python buildhistory_emit_pkghistory() {
>       rcpinfo.pv = pv
>       rcpinfo.pr = pr
>       rcpinfo.depends = sortlist(squashspaces(d.getVar('DEPENDS', True) or ""))
> +    rcpinfo.bbfile = bbfile
> +    rcpinfo.src_uri = src_uri
> +    rcpinfo.srcrev = srcrev
> +    rcpinfo.srcrev_autorev = srcrev_autorev
>       rcpinfo.packages = packages
>       write_recipehistory(rcpinfo, d)
>
> @@ -241,6 +256,12 @@ def write_recipehistory(rcpinfo, d):
>           f.write("DEPENDS = %s\n" %  rcpinfo.depends)
>           f.write("PACKAGES = %s\n" %  rcpinfo.packages)
>
> +    if rcpinfo.srcrev:
> +        srcrevfile = os.path.join(pkghistdir, "latest_srcrev")
> +        with open(srcrevfile, "w") as f:
> +            f.write(','.join([rcpinfo.bbfile, rcpinfo.src_uri, rcpinfo.srcrev,
> +                        rcpinfo.srcrev_autorev]))
> +
>
>   def write_pkghistory(pkginfo, d):
>       bb.debug(2, "Writing package history for package %s" % pkginfo.name)
>

I found the following problem with multiple builds:

> ERROR: The stack trace of python calls that resulted in this exception/failure was:
> ERROR:   File "buildhistory_emit_pkghistory", line 200, in <module>
> ERROR:
> ERROR:   File "buildhistory_emit_pkghistory", line 136, in buildhistory_emit_pkghistory
> ERROR:
> ERROR: The code that was being executed was:
> ERROR:      0196:
> ERROR:      0197:        write_pkghistory(pkginfo, d)
> ERROR:      0198:
> ERROR:      0199:
> ERROR:  *** 0200:buildhistory_emit_pkghistory(d)
> ERROR:      0201:
> ERROR: [From file: 'buildhistory_emit_pkghistory', lineno: 200, function: <module>]
> ERROR:      0132:        for item in os.listdir(pkghistdir):
> ERROR:      0133:            if item != "latest":
> ERROR:      0134:                if item not in packagelist:
> ERROR:      0135:                    subdir = os.path.join(pkghistdir, item)
> ERROR:  *** 0136:                    for subfile in os.listdir(subdir):
> ERROR:      0137:                        os.unlink(os.path.join(subdir, subfile))
> ERROR:      0138:                    os.rmdir(subdir)
> ERROR:      0139:
> ERROR:      0140:    rcpinfo = RecipeInfo(pn)
> ERROR: [From file: 'buildhistory_emit_pkghistory', lineno: 136, function: buildhistory_emit_pkghistory]
> ERROR: Function failed: buildhistory_emit_pkghistory
> ERROR: Logfile of failure stored in: /home/sgw/yocto/builds/world/tmp/work/x86_64-poky-linux/libgcc/4.7.2-r19/temp/log.do_package.24155
> ERROR: Task 953 (/home/sgw/yocto/poky/meta/recipes-devtools/gcc/libgcc_4.7.bb, do_package) failed with exit code '1'

It occurs with base-files also.

On 03/01/2013 09:29 PM, Saul Wold wrote:
> On 02/27/2013 07:37 AM, Constantin Musca wrote:
>> - create "latest_srcrev" for each recipe with the following format:
>> ${BB_FILENAME},${SRC_URI},${SRCREV},${FROM_AUTOREV}
>>
>> [YOCTO #3041]
>>
>> Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
>> ---
>>   meta/classes/buildhistory.bbclass | 23 ++++++++++++++++++++++-
>>   1 file changed, 22 insertions(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/buildhistory.bbclass 
>> b/meta/classes/buildhistory.bbclass
>> index a20d03d..fa13ef5 100644
>> --- a/meta/classes/buildhistory.bbclass
>> +++ b/meta/classes/buildhistory.bbclass
>> @@ -44,6 +44,11 @@ python buildhistory_emit_pkghistory() {
>>               self.pr = "r0"
>>               self.depends = ""
>>               self.packages = ""
>> +            self.bbfile = ""
>> +            self.src_uri = ""
>> +            self.srcrev = ""
>> +            self.srcrev_autorev = ""
>> +
>>
>>       class PackageInfo:
>>           def __init__(self, name):
>> @@ -151,6 +156,12 @@ python buildhistory_emit_pkghistory() {
>>       pe = d.getVar('PE', True) or "0"
>>       pv = d.getVar('PV', True)
>>       pr = d.getVar('PR', True)
>> +
>> +    bbfile = d.getVar('BB_FILENAME', True)
>> +    src_uri = d.getVar('SRC_URI', True)
>> +    srcrev = d.getVar('SRCREV', True)
>> +    srcrev_autorev = 'yes' if d.getVar('SRCREV', False) == 'AUTOINC' 
>> else 'no'
>> +
>>       packages = squashspaces(d.getVar('PACKAGES', True))
>>
>>       packagelist = packages.split()
>> @@ -159,7 +170,7 @@ python buildhistory_emit_pkghistory() {
>>       else:
>>           # Remove files for packages that no longer exist
>>           for item in os.listdir(pkghistdir):
>> -            if item != "latest":
>> +            if item != "latest" and item != "latest_srcrev":
>>                   if item not in packagelist:
>>                       subdir = os.path.join(pkghistdir, item)
>>                       for subfile in os.listdir(subdir):
>> @@ -171,6 +182,10 @@ python buildhistory_emit_pkghistory() {
>>       rcpinfo.pv = pv
>>       rcpinfo.pr = pr
>>       rcpinfo.depends = sortlist(squashspaces(d.getVar('DEPENDS', 
>> True) or ""))
>> +    rcpinfo.bbfile = bbfile
>> +    rcpinfo.src_uri = src_uri
>> +    rcpinfo.srcrev = srcrev
>> +    rcpinfo.srcrev_autorev = srcrev_autorev
>>       rcpinfo.packages = packages
>>       write_recipehistory(rcpinfo, d)
>>
>> @@ -241,6 +256,12 @@ def write_recipehistory(rcpinfo, d):
>>           f.write("DEPENDS = %s\n" %  rcpinfo.depends)
>>           f.write("PACKAGES = %s\n" %  rcpinfo.packages)
>>
>> +    if rcpinfo.srcrev:
>> +        srcrevfile = os.path.join(pkghistdir, "latest_srcrev")
>> +        with open(srcrevfile, "w") as f:
>> +            f.write(','.join([rcpinfo.bbfile, rcpinfo.src_uri, 
>> rcpinfo.srcrev,
>> +                        rcpinfo.srcrev_autorev]))
>> +
>>
>>   def write_pkghistory(pkginfo, d):
>>       bb.debug(2, "Writing package history for package %s" % 
>> pkginfo.name)
>>
>
> I found the following problem with multiple builds:
>
>> ERROR: The stack trace of python calls that resulted in this 
>> exception/failure was:
>> ERROR:   File "buildhistory_emit_pkghistory", line 200, in <module>
>> ERROR:
>> ERROR:   File "buildhistory_emit_pkghistory", line 136, in 
>> buildhistory_emit_pkghistory
>> ERROR:
>> ERROR: The code that was being executed was:
>> ERROR:      0196:
>> ERROR:      0197:        write_pkghistory(pkginfo, d)
>> ERROR:      0198:
>> ERROR:      0199:
>> ERROR:  *** 0200:buildhistory_emit_pkghistory(d)
>> ERROR:      0201:
>> ERROR: [From file: 'buildhistory_emit_pkghistory', lineno: 200, 
>> function: <module>]
>> ERROR:      0132:        for item in os.listdir(pkghistdir):
>> ERROR:      0133:            if item != "latest":

The patch changes this line to : if item != "latest" and item != 
"latest_srcrev":
I think the patch has not been applied correctly. I tested this locally 
and it worked.

Cheers,
Constantin


>> ERROR:      0134:                if item not in packagelist:
>> ERROR:      0135:                    subdir = 
>> os.path.join(pkghistdir, item)
>> ERROR:  *** 0136:                    for subfile in os.listdir(subdir):
>> ERROR:      0137: os.unlink(os.path.join(subdir, subfile))
>> ERROR:      0138:                    os.rmdir(subdir)
>> ERROR:      0139:
>> ERROR:      0140:    rcpinfo = RecipeInfo(pn)
>> ERROR: [From file: 'buildhistory_emit_pkghistory', lineno: 136, 
>> function: buildhistory_emit_pkghistory]
>> ERROR: Function failed: buildhistory_emit_pkghistory
>> ERROR: Logfile of failure stored in: 
>> /home/sgw/yocto/builds/world/tmp/work/x86_64-poky-linux/libgcc/4.7.2-r19/temp/log.do_package.24155
>> ERROR: Task 953 
>> (/home/sgw/yocto/poky/meta/recipes-devtools/gcc/libgcc_4.7.bb, 
>> do_package) failed with exit code '1'
>
> It occurs with base-files also.