Patchwork kernel: Use hardlinks for do_populate_sysroot for speed

login
register
mail settings
Submitter Richard Purdie
Date Nov. 8, 2013, 3:18 p.m.
Message ID <1383923880.2345.4.camel@ted>
Download mbox | patch
Permalink /patch/61331/
State New
Headers show

Comments

Richard Purdie - Nov. 8, 2013, 3:18 p.m.
The kernel tree is large and doesn't need to be copied. Override
the default sysroot handling function to use a hardlink copying
function in python.

This commit also drops the copying of the /lib directory which
just contains the kernel modules. We never use those in the sysroot
so there is little point in carrying those around.

For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
Bruce Ashfield - Nov. 8, 2013, 3:41 p.m.
On 13-11-08 10:18 AM, Richard Purdie wrote:
> The kernel tree is large and doesn't need to be copied. Override
> the default sysroot handling function to use a hardlink copying
> function in python.
>
> This commit also drops the copying of the /lib directory which
> just contains the kernel modules. We never use those in the sysroot
> so there is little point in carrying those around.
>
> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.

Fantastic. One less thing for me to dig into later. I thought this
was already in place, so I'm pleasantly surprised that there was a
time savings to be found!

Bruce

>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 0b6360f..e793ccb 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -308,8 +308,8 @@ kernelscripts_sstate_postinst () {
>   	fi
>   }
>
> -sysroot_stage_all_append() {
> -	sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}
> +python sysroot_stage_all () {
> +    oe.path.copyhardlinktree(d.expand("${D}${KERNEL_SRC_PATH}"), d.expand("${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}"))
>   }
>
>   kernel_do_configure() {
>
>
Hans Beckérus - Nov. 8, 2013, 3:54 p.m.
On Fri, Nov 8, 2013 at 4:41 PM, Bruce Ashfield
<bruce.ashfield@windriver.com> wrote:
> On 13-11-08 10:18 AM, Richard Purdie wrote:
>>
>> The kernel tree is large and doesn't need to be copied. Override
>> the default sysroot handling function to use a hardlink copying
>> function in python.
>>
>> This commit also drops the copying of the /lib directory which
>> just contains the kernel modules. We never use those in the sysroot
>> so there is little point in carrying those around.
>>
>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
>
>
> Fantastic. One less thing for me to dig into later. I thought this
> was already in place, so I'm pleasantly surprised that there was a
> time savings to be found!
>

+1
Looks really promising. Trying them out now in my Yocto dora branch.
Btw, would it make sense to group these into a patch set and then
number them appropriately?
I realized this when I was trying to apply them since I had to take
some care not mixing them up.

Thanks.
Hans


> Bruce
>
>
>>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> ---
>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>> index 0b6360f..e793ccb 100644
>> --- a/meta/classes/kernel.bbclass
>> +++ b/meta/classes/kernel.bbclass
>> @@ -308,8 +308,8 @@ kernelscripts_sstate_postinst () {
>>         fi
>>   }
>>
>> -sysroot_stage_all_append() {
>> -       sysroot_stage_dir ${D}${KERNEL_SRC_PATH}
>> ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}
>> +python sysroot_stage_all () {
>> +    oe.path.copyhardlinktree(d.expand("${D}${KERNEL_SRC_PATH}"),
>> d.expand("${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}"))
>>   }
>>
>>   kernel_do_configure() {
>>
>>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Richard Purdie - Nov. 8, 2013, 3:55 p.m.
On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
> On 13-11-08 10:18 AM, Richard Purdie wrote:
> > The kernel tree is large and doesn't need to be copied. Override
> > the default sysroot handling function to use a hardlink copying
> > function in python.
> >
> > This commit also drops the copying of the /lib directory which
> > just contains the kernel modules. We never use those in the sysroot
> > so there is little point in carrying those around.
> >
> > For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
> 
> Fantastic. One less thing for me to dig into later. I thought this
> was already in place, so I'm pleasantly surprised that there was a
> time savings to be found!

I was somewhat surprised too.

We still need to optimise what we install in do_install since that is
where significant gains can still be made.

Cheers,

Richard
Bruce Ashfield - Nov. 8, 2013, 3:59 p.m.
On 13-11-08 10:55 AM, Richard Purdie wrote:
> On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
>> On 13-11-08 10:18 AM, Richard Purdie wrote:
>>> The kernel tree is large and doesn't need to be copied. Override
>>> the default sysroot handling function to use a hardlink copying
>>> function in python.
>>>
>>> This commit also drops the copying of the /lib directory which
>>> just contains the kernel modules. We never use those in the sysroot
>>> so there is little point in carrying those around.
>>>
>>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
>>
>> Fantastic. One less thing for me to dig into later. I thought this
>> was already in place, so I'm pleasantly surprised that there was a
>> time savings to be found!
>
> I was somewhat surprised too.
>
> We still need to optimise what we install in do_install since that is
> where significant gains can still be made.

Agreed. I started some changes in that area right after ELC-e, I'll
try and get them out sooner rather than later.

Bruce

>
> Cheers,
>
> Richard
>
Richard Purdie - Nov. 8, 2013, 4:50 p.m.
On Fri, 2013-11-08 at 10:59 -0500, Bruce Ashfield wrote:
> On 13-11-08 10:55 AM, Richard Purdie wrote:
> > On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
> >> On 13-11-08 10:18 AM, Richard Purdie wrote:
> >>> The kernel tree is large and doesn't need to be copied. Override
> >>> the default sysroot handling function to use a hardlink copying
> >>> function in python.
> >>>
> >>> This commit also drops the copying of the /lib directory which
> >>> just contains the kernel modules. We never use those in the sysroot
> >>> so there is little point in carrying those around.
> >>>
> >>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
> >>
> >> Fantastic. One less thing for me to dig into later. I thought this
> >> was already in place, so I'm pleasantly surprised that there was a
> >> time savings to be found!
> >
> > I was somewhat surprised too.
> >
> > We still need to optimise what we install in do_install since that is
> > where significant gains can still be made.
> 
> Agreed. I started some changes in that area right after ELC-e, I'll
> try and get them out sooner rather than later.

I thought I'd share this for people's interest:

http://dan.rpsys.net/kernelbuildissue.png

Its the output from pybootchart of a bitbake core-image-sato from
scratch. I've zoomed out to put some bars in particular into
perspective.

The pink colour is linux-yocto:do_install, the cyan is
linux-yocto:do_package and the blue is linux-yocto:do_populate_sysroot.
The uncoloured bar at the bottom is linux-yocto:do_package_write_rpm.

So the final thing to build is the kernel by quite some margin, its
holding the rest of the build up.

Hopefully these patches start to improve that a bit!

Cheers,

Richard
Darren Hart - Nov. 8, 2013, 5:23 p.m.
On Fri, 2013-11-08 at 16:50 +0000, Richard Purdie wrote:
> On Fri, 2013-11-08 at 10:59 -0500, Bruce Ashfield wrote:
> > On 13-11-08 10:55 AM, Richard Purdie wrote:
> > > On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
> > >> On 13-11-08 10:18 AM, Richard Purdie wrote:
> > >>> The kernel tree is large and doesn't need to be copied. Override
> > >>> the default sysroot handling function to use a hardlink copying
> > >>> function in python.
> > >>>
> > >>> This commit also drops the copying of the /lib directory which
> > >>> just contains the kernel modules. We never use those in the sysroot
> > >>> so there is little point in carrying those around.
> > >>>
> > >>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
> > >>
> > >> Fantastic. One less thing for me to dig into later. I thought this
> > >> was already in place, so I'm pleasantly surprised that there was a
> > >> time savings to be found!
> > >
> > > I was somewhat surprised too.
> > >
> > > We still need to optimise what we install in do_install since that is
> > > where significant gains can still be made.
> > 
> > Agreed. I started some changes in that area right after ELC-e, I'll
> > try and get them out sooner rather than later.
> 
> I thought I'd share this for people's interest:
> 
> http://dan.rpsys.net/kernelbuildissue.png
> 
> Its the output from pybootchart of a bitbake core-image-sato from
> scratch. I've zoomed out to put some bars in particular into
> perspective.
> 
> The pink colour is linux-yocto:do_install, the cyan is
> linux-yocto:do_package and the blue is linux-yocto:do_populate_sysroot.
> The uncoloured bar at the bottom is linux-yocto:do_package_write_rpm.
> 
> So the final thing to build is the kernel by quite some margin, its
> holding the rest of the build up.
> 
> Hopefully these patches start to improve that a bit!
> 

Yup, linux-yocto do_package is the biggest build-time hindrance for me
and something we've wanted to look into for a long time. Great to see
some progress.
Andrea Adami - Nov. 9, 2013, 8:38 p.m.
On Fri, Nov 8, 2013 at 6:23 PM, Hart, Darren <darren.hart@intel.com> wrote:
> On Fri, 2013-11-08 at 16:50 +0000, Richard Purdie wrote:
>> On Fri, 2013-11-08 at 10:59 -0500, Bruce Ashfield wrote:
>> > On 13-11-08 10:55 AM, Richard Purdie wrote:
>> > > On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
>> > >> On 13-11-08 10:18 AM, Richard Purdie wrote:
>> > >>> The kernel tree is large and doesn't need to be copied. Override
>> > >>> the default sysroot handling function to use a hardlink copying
>> > >>> function in python.
>> > >>>
>> > >>> This commit also drops the copying of the /lib directory which
>> > >>> just contains the kernel modules. We never use those in the sysroot
>> > >>> so there is little point in carrying those around.
>> > >>>
>> > >>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
>> > >>
>> > >> Fantastic. One less thing for me to dig into later. I thought this
>> > >> was already in place, so I'm pleasantly surprised that there was a
>> > >> time savings to be found!
>> > >
>> > > I was somewhat surprised too.
>> > >
>> > > We still need to optimise what we install in do_install since that is
>> > > where significant gains can still be made.
>> >
>> > Agreed. I started some changes in that area right after ELC-e, I'll
>> > try and get them out sooner rather than later.
>>
>> I thought I'd share this for people's interest:
>>
>> http://dan.rpsys.net/kernelbuildissue.png
>>
>> Its the output from pybootchart of a bitbake core-image-sato from
>> scratch. I've zoomed out to put some bars in particular into
>> perspective.
>>
>> The pink colour is linux-yocto:do_install, the cyan is
>> linux-yocto:do_package and the blue is linux-yocto:do_populate_sysroot.
>> The uncoloured bar at the bottom is linux-yocto:do_package_write_rpm.
>>
>> So the final thing to build is the kernel by quite some margin, its
>> holding the rest of the build up.
>>
>> Hopefully these patches start to improve that a bit!
>>
>
> Yup, linux-yocto do_package is the biggest build-time hindrance for me
> and something we've wanted to look into for a long time. Great to see
> some progress.
>
> --
> Darren Hart
> Intel Open Source Technology Center
> Yocto Project - Linux Kernel
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

On my old quad core:

Before: 95.24 seconds
After: 83.25 seconds

Thanks

Andrea
Richard Purdie - Nov. 9, 2013, 10:58 p.m.
On Sat, 2013-11-09 at 21:38 +0100, Andrea Adami wrote:
> On Fri, Nov 8, 2013 at 6:23 PM, Hart, Darren <darren.hart@intel.com> wrote:
> > On Fri, 2013-11-08 at 16:50 +0000, Richard Purdie wrote:
> >> On Fri, 2013-11-08 at 10:59 -0500, Bruce Ashfield wrote:
> >> > On 13-11-08 10:55 AM, Richard Purdie wrote:
> >> > > On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
> >> > >> On 13-11-08 10:18 AM, Richard Purdie wrote:
> >> > >>> The kernel tree is large and doesn't need to be copied. Override
> >> > >>> the default sysroot handling function to use a hardlink copying
> >> > >>> function in python.
> >> > >>>
> >> > >>> This commit also drops the copying of the /lib directory which
> >> > >>> just contains the kernel modules. We never use those in the sysroot
> >> > >>> so there is little point in carrying those around.
> >> > >>>
> >> > >>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
> >> > >>
> >> > >> Fantastic. One less thing for me to dig into later. I thought this
> >> > >> was already in place, so I'm pleasantly surprised that there was a
> >> > >> time savings to be found!
> >> > >
> >> > > I was somewhat surprised too.
> >> > >
> >> > > We still need to optimise what we install in do_install since that is
> >> > > where significant gains can still be made.
> >> >
> >> > Agreed. I started some changes in that area right after ELC-e, I'll
> >> > try and get them out sooner rather than later.
> >>
> >> I thought I'd share this for people's interest:
> >>
> >> http://dan.rpsys.net/kernelbuildissue.png
> >>
> >> Its the output from pybootchart of a bitbake core-image-sato from
> >> scratch. I've zoomed out to put some bars in particular into
> >> perspective.
> >>
> >> The pink colour is linux-yocto:do_install, the cyan is
> >> linux-yocto:do_package and the blue is linux-yocto:do_populate_sysroot.
> >> The uncoloured bar at the bottom is linux-yocto:do_package_write_rpm.
> >>
> >> So the final thing to build is the kernel by quite some margin, its
> >> holding the rest of the build up.
> >>
> >> Hopefully these patches start to improve that a bit!
> >>
> >
> > Yup, linux-yocto do_package is the biggest build-time hindrance for me
> > and something we've wanted to look into for a long time. Great to see
> > some progress.
> >
> > --
> > Darren Hart
> > Intel Open Source Technology Center
> > Yocto Project - Linux Kernel
> >
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> On my old quad core:
> 
> Before: 95.24 seconds
> After: 83.25 seconds

We have some "standard" tests we use for comparison purposes as detailed
on:

https://wiki.yoctoproject.org/wiki/Performance_Test

Some numbers from master and a test branch containing the recent
speedups:

fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:22:37,12:05.50,1:17:53,4:02.68,0:31.71,0:15.70,0:01.61,26347744,5035816
fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:22:59,12:09.26,1:17:29,4:06.75,0:31.82,0:15.69,0:01.70,26348556,5035876
fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:22:21,12:07.23,1:18:21,4:07.44,0:31.79,0:15.63,0:01.62,26351088,5035908
fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:23:16,12:15.33,1:18:00,4:02.50,0:31.87,0:15.74,0:01.63,26348132,5035848
fedora19,rpurdie/timing:9d8c0ef3349936b3bd0bbf485b50cf81f4feaf80,poky-10.0.0.final-373-g9d8c0ef,1:19:42,10:28.53,1:15:30,4:12.24,0:31.30,0:15.69,0:01.66,26137344,5055628
fedora19,rpurdie/timing2:eb2221d6247c12322ac38b9ac6f24c9de5877e53,poky-10.0.0.final-401-geb2221d,1:18:51,10:44.26,1:16:32,4:13.47,0:31.27,0:15.69,0:01.65,26137828,5055916

So we have 1:23m -> 1:19m for overall buildtime and 12.0m -> 10.5m for
"bitbake virtual/kernel" from cleansstate. Not too bad :)

Thanks to Stefan for running the tests!

Cheers,

Richard
Hans Beckérus - Nov. 11, 2013, 8:06 a.m.
On Sat, Nov 9, 2013 at 11:58 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Sat, 2013-11-09 at 21:38 +0100, Andrea Adami wrote:
>> On Fri, Nov 8, 2013 at 6:23 PM, Hart, Darren <darren.hart@intel.com> wrote:
>> > On Fri, 2013-11-08 at 16:50 +0000, Richard Purdie wrote:
>> >> On Fri, 2013-11-08 at 10:59 -0500, Bruce Ashfield wrote:
>> >> > On 13-11-08 10:55 AM, Richard Purdie wrote:
>> >> > > On Fri, 2013-11-08 at 10:41 -0500, Bruce Ashfield wrote:
>> >> > >> On 13-11-08 10:18 AM, Richard Purdie wrote:
>> >> > >>> The kernel tree is large and doesn't need to be copied. Override
>> >> > >>> the default sysroot handling function to use a hardlink copying
>> >> > >>> function in python.
>> >> > >>>
>> >> > >>> This commit also drops the copying of the /lib directory which
>> >> > >>> just contains the kernel modules. We never use those in the sysroot
>> >> > >>> so there is little point in carrying those around.
>> >> > >>>
>> >> > >>> For linux-yocto this takes the do_populate_sysroot time 24s -> 14s.
>> >> > >>
>> >> > >> Fantastic. One less thing for me to dig into later. I thought this
>> >> > >> was already in place, so I'm pleasantly surprised that there was a
>> >> > >> time savings to be found!
>> >> > >
>> >> > > I was somewhat surprised too.
>> >> > >
>> >> > > We still need to optimise what we install in do_install since that is
>> >> > > where significant gains can still be made.
>> >> >
>> >> > Agreed. I started some changes in that area right after ELC-e, I'll
>> >> > try and get them out sooner rather than later.
>> >>
>> >> I thought I'd share this for people's interest:
>> >>
>> >> http://dan.rpsys.net/kernelbuildissue.png
>> >>
>> >> Its the output from pybootchart of a bitbake core-image-sato from
>> >> scratch. I've zoomed out to put some bars in particular into
>> >> perspective.
>> >>
>> >> The pink colour is linux-yocto:do_install, the cyan is
>> >> linux-yocto:do_package and the blue is linux-yocto:do_populate_sysroot.
>> >> The uncoloured bar at the bottom is linux-yocto:do_package_write_rpm.
>> >>
>> >> So the final thing to build is the kernel by quite some margin, its
>> >> holding the rest of the build up.
>> >>
>> >> Hopefully these patches start to improve that a bit!
>> >>
>> >
>> > Yup, linux-yocto do_package is the biggest build-time hindrance for me
>> > and something we've wanted to look into for a long time. Great to see
>> > some progress.
>> >
>> > --
>> > Darren Hart
>> > Intel Open Source Technology Center
>> > Yocto Project - Linux Kernel
>> >
>> > _______________________________________________
>> > Openembedded-core mailing list
>> > Openembedded-core@lists.openembedded.org
>> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>> On my old quad core:
>>
>> Before: 95.24 seconds
>> After: 83.25 seconds
>
> We have some "standard" tests we use for comparison purposes as detailed
> on:
>
> https://wiki.yoctoproject.org/wiki/Performance_Test
>
> Some numbers from master and a test branch containing the recent
> speedups:
>
> fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:22:37,12:05.50,1:17:53,4:02.68,0:31.71,0:15.70,0:01.61,26347744,5035816
> fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:22:59,12:09.26,1:17:29,4:06.75,0:31.82,0:15.69,0:01.70,26348556,5035876
> fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:22:21,12:07.23,1:18:21,4:07.44,0:31.79,0:15.63,0:01.62,26351088,5035908
> fedora19,master:d6cc7c8ed76c8b1117cf03c7bd4b0742f98f79b3,poky-10.0.0.final-359-gd6cc7c8,1:23:16,12:15.33,1:18:00,4:02.50,0:31.87,0:15.74,0:01.63,26348132,5035848
> fedora19,rpurdie/timing:9d8c0ef3349936b3bd0bbf485b50cf81f4feaf80,poky-10.0.0.final-373-g9d8c0ef,1:19:42,10:28.53,1:15:30,4:12.24,0:31.30,0:15.69,0:01.66,26137344,5055628
> fedora19,rpurdie/timing2:eb2221d6247c12322ac38b9ac6f24c9de5877e53,poky-10.0.0.final-401-geb2221d,1:18:51,10:44.26,1:16:32,4:13.47,0:31.27,0:15.69,0:01.65,26137828,5055916
>
> So we have 1:23m -> 1:19m for overall buildtime and 12.0m -> 10.5m for
> "bitbake virtual/kernel" from cleansstate. Not too bad :)
>
> Thanks to Stefan for running the tests!
>

I am having some trouble with these patches :( The new approach is
using the '-n' flag to the 'cp' command.
That is not supported on our SuSE11 based system. To be honest, I do
not know how portable '-n' is?
Some system have it, others seems to provide '-u' instead. I guess by
removing '-n' part of the performance gain is lost?
Would it be possible to test for error from the 'cp' command and if it
fails try '-u' instead (and cache the result)? Or maybe
even better, make this configurable in local.conf (or the distro) for
the build platforms that does not support  '-n' but can
instead fall-back to using '-u'. From what I can tell most systems
support  one or the other, but never both.

Thanks.
Hans


> Cheers,
>
> Richard
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Richard Purdie - Nov. 11, 2013, 9:35 a.m.
On Mon, 2013-11-11 at 09:06 +0100, Hans Beckérus wrote:
> I am having some trouble with these patches :( The new approach is
> using the '-n' flag to the 'cp' command.
> That is not supported on our SuSE11 based system. To be honest, I do
> not know how portable '-n' is?
> Some system have it, others seems to provide '-u' instead. I guess by
> removing '-n' part of the performance gain is lost?
> Would it be possible to test for error from the 'cp' command and if it
> fails try '-u' instead (and cache the result)? Or maybe
> even better, make this configurable in local.conf (or the distro) for
> the build platforms that does not support  '-n' but can
> instead fall-back to using '-u'. From what I can tell most systems
> support  one or the other, but never both.

Hmm, its been in coreutils since 2009 which I guess isn't that long in
the scheme of things:
http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/cp.c?id=d01338eb3d30e5634f1b4d4179c229f54eea0b44

Just to double check, is cp on your system provided by coreutils or
something else?

Cheers,

Richard
Richard Purdie - Nov. 11, 2013, 9:45 a.m.
On Mon, 2013-11-11 at 09:06 +0100, Hans Beckérus wrote:
> I am having some trouble with these patches :( The new approach is
> using the '-n' flag to the 'cp' command.
> That is not supported on our SuSE11 based system. To be honest, I do
> not know how portable '-n' is?
> Some system have it, others seems to provide '-u' instead. I guess by
> removing '-n' part of the performance gain is lost?
> Would it be possible to test for error from the 'cp' command and if it
> fails try '-u' instead (and cache the result)? Or maybe
> even better, make this configurable in local.conf (or the distro) for
> the build platforms that does not support  '-n' but can
> instead fall-back to using '-u'. From what I can tell most systems
> support  one or the other, but never both.

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=276b9df3588ecd438c3e2f502d56cd6b9fb24676

Thankfully the above works just as well. I used -n to get rid of an
error, then added -l for efficiency. We can get away just with -l.

Cheers,

Richard
Hans Beckérus - Nov. 11, 2013, 9:47 a.m.
On Mon, Nov 11, 2013 at 10:45 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Mon, 2013-11-11 at 09:06 +0100, Hans Beckérus wrote:
>> I am having some trouble with these patches :( The new approach is
>> using the '-n' flag to the 'cp' command.
>> That is not supported on our SuSE11 based system. To be honest, I do
>> not know how portable '-n' is?
>> Some system have it, others seems to provide '-u' instead. I guess by
>> removing '-n' part of the performance gain is lost?
>> Would it be possible to test for error from the 'cp' command and if it
>> fails try '-u' instead (and cache the result)? Or maybe
>> even better, make this configurable in local.conf (or the distro) for
>> the build platforms that does not support  '-n' but can
>> instead fall-back to using '-u'. From what I can tell most systems
>> support  one or the other, but never both.
>
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t2&id=276b9df3588ecd438c3e2f502d56cd6b9fb24676
>
> Thankfully the above works just as well. I used -n to get rid of an
> error, then added -l for efficiency. We can get away just with -l.
>
Great! Will try it asap.
Btw, cp was from coreutils back in 2008.

cp (GNU coreutils) 6.12
Copyright (C) 2008 Free Software Foundation, Inc.

Thanks.
Hans

> Cheers,
>
> Richard
>
>

Patch

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 0b6360f..e793ccb 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -308,8 +308,8 @@  kernelscripts_sstate_postinst () {
 	fi
 }
 
-sysroot_stage_all_append() {
-	sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}
+python sysroot_stage_all () {
+    oe.path.copyhardlinktree(d.expand("${D}${KERNEL_SRC_PATH}"), d.expand("${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}"))
 }
 
 kernel_do_configure() {