Patchwork [0/6] Integrate Smart for RPM image generation

login
register
mail settings
Submitter Mark Hatle
Date Dec. 4, 2012, 7:49 p.m.
Message ID <cover.1354650148.git.mark.hatle@windriver.com>
Download mbox
Permalink /patch/40355/
State New
Headers show

Pull-request

git://git.yoctoproject.org/poky-contrib mhatle/smart

Comments

Mark Hatle - Dec. 4, 2012, 7:49 p.m.
The following patch set relies on the previous "Misc fixes" sent out earlier
today.

The set replaces the existing rootfs generation with one based on Smart.
The work was tested using buildhistory over a wide range of image types:
core-image-minimal
core-image-base
core-image-basic
core-image-lsb
core-image-sato
build-appliance

Each of which contained minor changes from the former version, but the changes
can all be explained by a different package resolver.

Multilib configurations of each of the above images were also verified on 
qemux86_64 configuration with a few lib32 items added to the mix.  This was
shown, by buildhistory, to result in a more compact install solution.

"populate_sdk" was also verified after this patch.  Buildhistory does not
verify SDKs, so these were verified by hand.  The target items all appeared
to match the expected image components.

The incremental RPM installation code was verified as well.. This code was 
verified in both multilib and non-multilib configurations.

Finally, each of these were verified on: qemux86_64, qemumips, qemuppc, qemuarm,
and qemuarm configured w/ hardware float API.  This verifies that both the 
ARM specific "gnueabi" issues and the _ to - in the arch names both work 
properly.

The following changes since commit 0c69e555dda2fa412ba12fa90886b0b92dc60e44:

  license.bbclass: Avoid grep error message (2012-12-04 11:10:18 -0600)

are available in the git repository at:
  git://git.yoctoproject.org/poky-contrib mhatle/smart
  http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/smart

Mark Hatle (4):
  rootfs_rpm.bbclass: Revert to default shell logging behavior
  package_rpm: Update the way the multilib package names are translated
  package_rpm.bbclass: Add additional logging
  package_rpm.bbclass: Add support for incremental installs

Paul Eggleton (2):
  classes/*_rpm: integrate Smart into RPM filesystem construction
  classes/rootfs_rpm: install smart instead of zypper in rpm-based
    images

 meta/classes/package_rpm.bbclass      |  691 +++++++++++++--------------------
 meta/classes/populate_sdk_rpm.bbclass |   74 ++---
 meta/classes/rootfs_rpm.bbclass       |   86 ++---
 3 files changed, 338 insertions(+), 513 deletions(-)
Saul Wold - Dec. 6, 2012, 5:08 a.m.
On 12/04/2012 11:49 AM, Mark Hatle wrote:
> The following patch set relies on the previous "Misc fixes" sent out earlier
> today.
>
> The set replaces the existing rootfs generation with one based on Smart.
> The work was tested using buildhistory over a wide range of image types:
> core-image-minimal
> core-image-base
> core-image-basic
> core-image-lsb
> core-image-sato
> build-appliance
>
Did you try with the DISTRO = poky-lsb when doing the lsb build?  There 
are failures across the board with the LSB builds, see this example:

http://autobuilder.yoctoproject.org:8010/builders/nightly-x86-64-lsb/builds/146/steps/shell_29/logs/stdio


The failure seems to be on psplash:
| Error: psplash not found in the base feeds (qemux86_64 x86_64 noarch 
any all).
| ERROR: Function failed: do_rootfs (see 
/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86-64-lsb/build/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-dev/1.0-r0/temp/log.do_rootfs.10377 
for further information)
NOTE: recipe core-image-lsb-dev-1.0-r0: task do_rootfs: Failed

Can you look into this before we integrate into master

Thanks
	Sau!



> Each of which contained minor changes from the former version, but the changes
> can all be explained by a different package resolver.
>
> Multilib configurations of each of the above images were also verified on
> qemux86_64 configuration with a few lib32 items added to the mix.  This was
> shown, by buildhistory, to result in a more compact install solution.
>
> "populate_sdk" was also verified after this patch.  Buildhistory does not
> verify SDKs, so these were verified by hand.  The target items all appeared
> to match the expected image components.
>
> The incremental RPM installation code was verified as well.. This code was
> verified in both multilib and non-multilib configurations.
>
> Finally, each of these were verified on: qemux86_64, qemumips, qemuppc, qemuarm,
> and qemuarm configured w/ hardware float API.  This verifies that both the
> ARM specific "gnueabi" issues and the _ to - in the arch names both work
> properly.
>
> The following changes since commit 0c69e555dda2fa412ba12fa90886b0b92dc60e44:
>
>    license.bbclass: Avoid grep error message (2012-12-04 11:10:18 -0600)
>
> are available in the git repository at:
>    git://git.yoctoproject.org/poky-contrib mhatle/smart
>    http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/smart
>
> Mark Hatle (4):
>    rootfs_rpm.bbclass: Revert to default shell logging behavior
>    package_rpm: Update the way the multilib package names are translated
>    package_rpm.bbclass: Add additional logging
>    package_rpm.bbclass: Add support for incremental installs
>
> Paul Eggleton (2):
>    classes/*_rpm: integrate Smart into RPM filesystem construction
>    classes/rootfs_rpm: install smart instead of zypper in rpm-based
>      images
>
>   meta/classes/package_rpm.bbclass      |  691 +++++++++++++--------------------
>   meta/classes/populate_sdk_rpm.bbclass |   74 ++---
>   meta/classes/rootfs_rpm.bbclass       |   86 ++---
>   3 files changed, 338 insertions(+), 513 deletions(-)
>
Paul Eggleton - Dec. 6, 2012, 1:58 p.m.
On Wednesday 05 December 2012 21:08:59 Saul Wold wrote:
> On 12/04/2012 11:49 AM, Mark Hatle wrote:
> > The following patch set relies on the previous "Misc fixes" sent out
> > earlier today.
> > 
> > The set replaces the existing rootfs generation with one based on Smart.
> > The work was tested using buildhistory over a wide range of image types:
> > core-image-minimal
> > core-image-base
> > core-image-basic
> > core-image-lsb
> > core-image-sato
> > build-appliance
> 
> Did you try with the DISTRO = poky-lsb when doing the lsb build?  There
> are failures across the board with the LSB builds

I just did with core-image-lsb, and apart from the pkgconfig issue which I have 
worked around, it built fine - I did not see this error. The only thing I can 
think of is that it has something to do with these running concurrently, 
although I'm not sure how that could be a problem especially as we have 
locking for the rpm backend.

I'm now running a concurrent build of all of the four images that failed, 
we'll see if I get the same failure or not fairly soon.

Cheers,
Paul
Mark Hatle - Dec. 6, 2012, 5:04 p.m.
On 12/5/12 11:08 PM, Saul Wold wrote:
> On 12/04/2012 11:49 AM, Mark Hatle wrote:
>> The following patch set relies on the previous "Misc fixes" sent out earlier
>> today.
>>
>> The set replaces the existing rootfs generation with one based on Smart.
>> The work was tested using buildhistory over a wide range of image types:
>> core-image-minimal
>> core-image-base
>> core-image-basic
>> core-image-lsb
>> core-image-sato
>> build-appliance
>>
> Did you try with the DISTRO = poky-lsb when doing the lsb build?  There
> are failures across the board with the LSB builds, see this example:

There is no poky-lsb within oe-core, so no I did not.

Do you have a branch with this integrated into Poky, if so I can try to 
reproduce it locally.

--Mark

> http://autobuilder.yoctoproject.org:8010/builders/nightly-x86-64-lsb/builds/146/steps/shell_29/logs/stdio
>
>
> The failure seems to be on psplash:
> | Error: psplash not found in the base feeds (qemux86_64 x86_64 noarch
> any all).
> | ERROR: Function failed: do_rootfs (see
> /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86-64-lsb/build/build/tmp/work/qemux86_64-poky-linux/core-image-lsb-dev/1.0-r0/temp/log.do_rootfs.10377
> for further information)
> NOTE: recipe core-image-lsb-dev-1.0-r0: task do_rootfs: Failed
>
> Can you look into this before we integrate into master
>
> Thanks
> 	Sau!
>
>
>
>> Each of which contained minor changes from the former version, but the changes
>> can all be explained by a different package resolver.
>>
>> Multilib configurations of each of the above images were also verified on
>> qemux86_64 configuration with a few lib32 items added to the mix.  This was
>> shown, by buildhistory, to result in a more compact install solution.
>>
>> "populate_sdk" was also verified after this patch.  Buildhistory does not
>> verify SDKs, so these were verified by hand.  The target items all appeared
>> to match the expected image components.
>>
>> The incremental RPM installation code was verified as well.. This code was
>> verified in both multilib and non-multilib configurations.
>>
>> Finally, each of these were verified on: qemux86_64, qemumips, qemuppc, qemuarm,
>> and qemuarm configured w/ hardware float API.  This verifies that both the
>> ARM specific "gnueabi" issues and the _ to - in the arch names both work
>> properly.
>>
>> The following changes since commit 0c69e555dda2fa412ba12fa90886b0b92dc60e44:
>>
>>     license.bbclass: Avoid grep error message (2012-12-04 11:10:18 -0600)
>>
>> are available in the git repository at:
>>     git://git.yoctoproject.org/poky-contrib mhatle/smart
>>     http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/smart
>>
>> Mark Hatle (4):
>>     rootfs_rpm.bbclass: Revert to default shell logging behavior
>>     package_rpm: Update the way the multilib package names are translated
>>     package_rpm.bbclass: Add additional logging
>>     package_rpm.bbclass: Add support for incremental installs
>>
>> Paul Eggleton (2):
>>     classes/*_rpm: integrate Smart into RPM filesystem construction
>>     classes/rootfs_rpm: install smart instead of zypper in rpm-based
>>       images
>>
>>    meta/classes/package_rpm.bbclass      |  691 +++++++++++++--------------------
>>    meta/classes/populate_sdk_rpm.bbclass |   74 ++---
>>    meta/classes/rootfs_rpm.bbclass       |   86 ++---
>>    3 files changed, 338 insertions(+), 513 deletions(-)
>>
Paul Eggleton - Dec. 6, 2012, 5:22 p.m.
On Thursday 06 December 2012 11:04:38 Mark Hatle wrote:
> On 12/5/12 11:08 PM, Saul Wold wrote:
> > On 12/04/2012 11:49 AM, Mark Hatle wrote:
> >> The following patch set relies on the previous "Misc fixes" sent out
> >> earlier today.
> >> 
> >> The set replaces the existing rootfs generation with one based on Smart.
> >> The work was tested using buildhistory over a wide range of image types:
> >> core-image-minimal
> >> core-image-base
> >> core-image-basic
> >> core-image-lsb
> >> core-image-sato
> >> build-appliance
> > 
> > Did you try with the DISTRO = poky-lsb when doing the lsb build?  There
> 
> > are failures across the board with the LSB builds, see this example:
> There is no poky-lsb within oe-core, so no I did not.
> 
> Do you have a branch with this integrated into Poky, if so I can try to
> reproduce it locally.

I've just updated the poky-contrib paule/smart branch.

Cheers,
Paul
Saul Wold - Dec. 8, 2012, 3:29 a.m.
On 12/04/2012 11:49 AM, Mark Hatle wrote:
> The following patch set relies on the previous "Misc fixes" sent out earlier
> today.
>
> The set replaces the existing rootfs generation with one based on Smart.
> The work was tested using buildhistory over a wide range of image types:
> core-image-minimal
> core-image-base
> core-image-basic
> core-image-lsb
> core-image-sato
> build-appliance
>
> Each of which contained minor changes from the former version, but the changes
> can all be explained by a different package resolver.
>
> Multilib configurations of each of the above images were also verified on
> qemux86_64 configuration with a few lib32 items added to the mix.  This was
> shown, by buildhistory, to result in a more compact install solution.
>
> "populate_sdk" was also verified after this patch.  Buildhistory does not
> verify SDKs, so these were verified by hand.  The target items all appeared
> to match the expected image components.
>
> The incremental RPM installation code was verified as well.. This code was
> verified in both multilib and non-multilib configurations.
>
> Finally, each of these were verified on: qemux86_64, qemumips, qemuppc, qemuarm,
> and qemuarm configured w/ hardware float API.  This verifies that both the
> ARM specific "gnueabi" issues and the _ to - in the arch names both work
> properly.
>

Sanity tests need to be written and exchanged for the now failing zypper 
sanity tests!!

Sau!

> The following changes since commit 0c69e555dda2fa412ba12fa90886b0b92dc60e44:
>
>    license.bbclass: Avoid grep error message (2012-12-04 11:10:18 -0600)
>
> are available in the git repository at:
>    git://git.yoctoproject.org/poky-contrib mhatle/smart
>    http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/smart
>
> Mark Hatle (4):
>    rootfs_rpm.bbclass: Revert to default shell logging behavior
>    package_rpm: Update the way the multilib package names are translated
>    package_rpm.bbclass: Add additional logging
>    package_rpm.bbclass: Add support for incremental installs
>
> Paul Eggleton (2):
>    classes/*_rpm: integrate Smart into RPM filesystem construction
>    classes/rootfs_rpm: install smart instead of zypper in rpm-based
>      images
>
>   meta/classes/package_rpm.bbclass      |  691 +++++++++++++--------------------
>   meta/classes/populate_sdk_rpm.bbclass |   74 ++---
>   meta/classes/rootfs_rpm.bbclass       |   86 ++---
>   3 files changed, 338 insertions(+), 513 deletions(-)
>
Saul Wold - Dec. 8, 2012, 6:37 a.m.
There still seems to be a problem with the multilib build that I believe 
comes from this set of changes.

> | Running intercept scripts:
> | WARNING: The license listed MIT was not in the licenses collected for lib32-packagegroup-core-boot
> | ls: cannot access /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-multilib/build/build/tmp/pkgdata/*/runtime-reverse/udev-extraconf: No such file or directory
> | readlink: missing operand
> | Try `readlink --help' for more information.
> | basename: missing operand
> | Try `basename --help' for more information.
> | ERROR: Function failed: do_rootfs (see /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-multilib/build/build/tmp/work/qemux86_64-poky-linux/lib32-core-image-minimal/1.0-r0/temp/log.do_rootfs.14393 for further information)
> NOTE: recipe lib32-core-image-minimal-1.0-r0: task do_rootfs: Failed
> ERROR: Task 7 (virtual:multilib:lib32:/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-multilib/build/meta/recipes-core/images/core-image-minimal.bb, do_rootfs) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 1867 tasks of which 425 didn't need to be rerun and 1 failed.

Sau!


On 12/04/2012 11:49 AM, Mark Hatle wrote:
> The following patch set relies on the previous "Misc fixes" sent out earlier
> today.
>
> The set replaces the existing rootfs generation with one based on Smart.
> The work was tested using buildhistory over a wide range of image types:
> core-image-minimal
> core-image-base
> core-image-basic
> core-image-lsb
> core-image-sato
> build-appliance
>
> Each of which contained minor changes from the former version, but the changes
> can all be explained by a different package resolver.
>
> Multilib configurations of each of the above images were also verified on
> qemux86_64 configuration with a few lib32 items added to the mix.  This was
> shown, by buildhistory, to result in a more compact install solution.
>
> "populate_sdk" was also verified after this patch.  Buildhistory does not
> verify SDKs, so these were verified by hand.  The target items all appeared
> to match the expected image components.
>
> The incremental RPM installation code was verified as well.. This code was
> verified in both multilib and non-multilib configurations.
>
> Finally, each of these were verified on: qemux86_64, qemumips, qemuppc, qemuarm,
> and qemuarm configured w/ hardware float API.  This verifies that both the
> ARM specific "gnueabi" issues and the _ to - in the arch names both work
> properly.
>
> The following changes since commit 0c69e555dda2fa412ba12fa90886b0b92dc60e44:
>
>    license.bbclass: Avoid grep error message (2012-12-04 11:10:18 -0600)
>
> are available in the git repository at:
>    git://git.yoctoproject.org/poky-contrib mhatle/smart
>    http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/smart
>
> Mark Hatle (4):
>    rootfs_rpm.bbclass: Revert to default shell logging behavior
>    package_rpm: Update the way the multilib package names are translated
>    package_rpm.bbclass: Add additional logging
>    package_rpm.bbclass: Add support for incremental installs
>
> Paul Eggleton (2):
>    classes/*_rpm: integrate Smart into RPM filesystem construction
>    classes/rootfs_rpm: install smart instead of zypper in rpm-based
>      images
>
>   meta/classes/package_rpm.bbclass      |  691 +++++++++++++--------------------
>   meta/classes/populate_sdk_rpm.bbclass |   74 ++---
>   meta/classes/rootfs_rpm.bbclass       |   86 ++---
>   3 files changed, 338 insertions(+), 513 deletions(-)
>
Mark Hatle - Dec. 8, 2012, 5:10 p.m.
What is the build configuration for this chunk?

Also this looks like it might be something within the license class, not the 
rootfs, but I'm not sure.

--Mark

On 12/8/12 12:37 AM, Saul Wold wrote:
>
> There still seems to be a problem with the multilib build that I believe
> comes from this set of changes.
>
>> | Running intercept scripts:
>> | WARNING: The license listed MIT was not in the licenses collected for lib32-packagegroup-core-boot
>> | ls: cannot access /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-multilib/build/build/tmp/pkgdata/*/runtime-reverse/udev-extraconf: No such file or directory
>> | readlink: missing operand
>> | Try `readlink --help' for more information.
>> | basename: missing operand
>> | Try `basename --help' for more information.
>> | ERROR: Function failed: do_rootfs (see /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-multilib/build/build/tmp/work/qemux86_64-poky-linux/lib32-core-image-minimal/1.0-r0/temp/log.do_rootfs.14393 for further information)
>> NOTE: recipe lib32-core-image-minimal-1.0-r0: task do_rootfs: Failed
>> ERROR: Task 7 (virtual:multilib:lib32:/srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-multilib/build/meta/recipes-core/images/core-image-minimal.bb, do_rootfs) failed with exit code '1'
>> NOTE: Tasks Summary: Attempted 1867 tasks of which 425 didn't need to be rerun and 1 failed.
>
> Sau!
>
>
> On 12/04/2012 11:49 AM, Mark Hatle wrote:
>> The following patch set relies on the previous "Misc fixes" sent out earlier
>> today.
>>
>> The set replaces the existing rootfs generation with one based on Smart.
>> The work was tested using buildhistory over a wide range of image types:
>> core-image-minimal
>> core-image-base
>> core-image-basic
>> core-image-lsb
>> core-image-sato
>> build-appliance
>>
>> Each of which contained minor changes from the former version, but the changes
>> can all be explained by a different package resolver.
>>
>> Multilib configurations of each of the above images were also verified on
>> qemux86_64 configuration with a few lib32 items added to the mix.  This was
>> shown, by buildhistory, to result in a more compact install solution.
>>
>> "populate_sdk" was also verified after this patch.  Buildhistory does not
>> verify SDKs, so these were verified by hand.  The target items all appeared
>> to match the expected image components.
>>
>> The incremental RPM installation code was verified as well.. This code was
>> verified in both multilib and non-multilib configurations.
>>
>> Finally, each of these were verified on: qemux86_64, qemumips, qemuppc, qemuarm,
>> and qemuarm configured w/ hardware float API.  This verifies that both the
>> ARM specific "gnueabi" issues and the _ to - in the arch names both work
>> properly.
>>
>> The following changes since commit 0c69e555dda2fa412ba12fa90886b0b92dc60e44:
>>
>>     license.bbclass: Avoid grep error message (2012-12-04 11:10:18 -0600)
>>
>> are available in the git repository at:
>>     git://git.yoctoproject.org/poky-contrib mhatle/smart
>>     http://git.yoctoproject.org/cgit.cgi//log/?h=mhatle/smart
>>
>> Mark Hatle (4):
>>     rootfs_rpm.bbclass: Revert to default shell logging behavior
>>     package_rpm: Update the way the multilib package names are translated
>>     package_rpm.bbclass: Add additional logging
>>     package_rpm.bbclass: Add support for incremental installs
>>
>> Paul Eggleton (2):
>>     classes/*_rpm: integrate Smart into RPM filesystem construction
>>     classes/rootfs_rpm: install smart instead of zypper in rpm-based
>>       images
>>
>>    meta/classes/package_rpm.bbclass      |  691 +++++++++++++--------------------
>>    meta/classes/populate_sdk_rpm.bbclass |   74 ++---
>>    meta/classes/rootfs_rpm.bbclass       |   86 ++---
>>    3 files changed, 338 insertions(+), 513 deletions(-)
>>