| Submitter | Khem Raj |
|---|---|
| Date | July 10, 2012, 5:07 p.m. |
| Message ID | <1341940070-27715-2-git-send-email-raj.khem@gmail.com> |
| Download | mbox | patch |
| Permalink | /patch/31693/ |
| State | New |
| Headers | show |
Comments
On Tue, 2012-07-10 at 10:07 -0700, Khem Raj wrote: > We shave too much from kernel sources for making it work > for on device external kernel module development that cross > development of external modules wont work from same tree > anymore. This patch makes a copy of tree which will eventually > be staged for building external modules Why does cross development require more of the kernel tree than on-device compilation? p.
On Tue, Jul 10, 2012 at 1:13 PM, Phil Blundell <philb@gnu.org> wrote: > On Tue, 2012-07-10 at 10:07 -0700, Khem Raj wrote: >> We shave too much from kernel sources for making it work >> for on device external kernel module development that cross >> development of external modules wont work from same tree >> anymore. This patch makes a copy of tree which will eventually >> be staged for building external modules > > Why does cross development require more of the kernel tree than > on-device compilation? From what we were discussing on IRC yesterday it is the binary tools that are being stripped (and then regenerated) on the on-target builds. We can't package them up, since they are in a -dev package. Cheers, Bruce > > p. > > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Tue, 2012-07-10 at 10:07 -0700, Khem Raj wrote: > We shave too much from kernel sources for making it work > for on device external kernel module development that cross > development of external modules wont work from same tree > anymore. This patch makes a copy of tree which will eventually > be staged for building external modules It sounds from the further discussion that there is more to the patch than just this. Firstly, a change like this needs a more precise description. Secondly, we're copying around *way* to much data in this approach. I'd like to suggest an improvement but can't since the description above is lacking, I can't even understand what the problem is you're trying to fix. So more discussion is needed. I have a suspicion that this fix only "happens" to work when SDKMACHINE == NATIVEMACHINE. Cheers, Richard > Signed-off-by: Khem Raj <raj.khem@gmail.com> > --- > meta/classes/kernel.bbclass | 15 +++++++++++++-- > 1 files changed, 13 insertions(+), 2 deletions(-) > > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass > index 81c334b..b2b6fcd 100644 > --- a/meta/classes/kernel.bbclass > +++ b/meta/classes/kernel.bbclass > @@ -80,6 +80,7 @@ KERNEL_ALT_IMAGETYPE ??= "" > # Define where the kernel headers are installed on the target as well as where > # they are staged. > KERNEL_SRC_PATH = "/usr/src/kernel" > +UNPRUNED_KERNEL_SRC_PATH = "/usr/src/kernel-unpruned" > > KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}" > > @@ -135,7 +136,9 @@ kernel_do_install() { > # kernel source tree. > # > kerneldir=${D}${KERNEL_SRC_PATH} > + unrprunedkerneldir=${D}${UNPRUNED_KERNEL_SRC_PATH} > install -d $kerneldir > + install -d $unrprunedkerneldir > > # > # Store the kernel version in sysroots for module-base.bbclass > @@ -176,7 +179,6 @@ kernel_do_install() { > # and include files. > # > oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean > - make -C $kerneldir _mrproper_scripts > find $kerneldir -path $kerneldir/lib -prune -o -path $kerneldir/tools -prune -o -path $kerneldir/scripts -prune -o -name "*.[csS]" -exec rm '{}' \; > find $kerneldir/Documentation -name "*.txt" -exec rm '{}' \; > > @@ -186,6 +188,9 @@ kernel_do_install() { > if [ ${ARCH} = "powerpc" ]; then > cp arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o > fi > + # make a copy of unpruned kernel tree > + cp -fR $kerneldir/* $unrprunedkerneldir > + make -C $kerneldir _mrproper_scripts > > # Remove the following binaries which cause strip or arch QA errors > # during do_package for cross-compiled platforms > @@ -197,8 +202,14 @@ kernel_do_install() { > done > } > > +PACKAGE_PREPROCESS_FUNCS += "kernel_package_preprocess" > + > +kernel_package_preprocess () { > + rm -rf ${PKGD}/${UNPRUNED_KERNEL_SRC_PATH} > +} > + > sysroot_stage_all_append() { > - sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} > + sysroot_stage_dir ${D}${UNPRUNED_KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} > } > > kernel_do_configure() {
On Wed, Jul 11, 2012 at 3:30 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > On Tue, 2012-07-10 at 10:07 -0700, Khem Raj wrote: >> We shave too much from kernel sources for making it work >> for on device external kernel module development that cross >> development of external modules wont work from same tree >> anymore. This patch makes a copy of tree which will eventually >> be staged for building external modules > > It sounds from the further discussion that there is more to the patch > than just this. Firstly, a change like this needs a more precise > description. > > Secondly, we're copying around *way* to much data in this approach. I'd > like to suggest an improvement but can't since the description above is > lacking, I can't even understand what the problem is you're trying to > fix. alright. There are tools (especially in scripts/) which are made for buildhost when compiling the kernel irrespective of cross or native. The tools like fixdep, recordmcount etc. which are needed to run when you build kernel itself as well as external modules. Now we do 'make _mproper_scripts' which cleans all those binaries. We do this because we want to ship kernel-dev and packaging binaries for different host wont be allowed in a target package. So we chose to delete them and then on device regenerate them ( ideally I would have preferred to generate them cross candian when making for target) Deleting these tools also renders the cross building of modules useless. Since I want to ship the sources as reference only meaning people may not have write access to the kernel sources they can not run make inside the kernel sources to regenerate those binaries before they start building their external modules. > > So more discussion is needed. > > I have a suspicion that this fix only "happens" to work when SDKMACHINE > == NATIVEMACHINE. > yes, for a full solution we have to generate two versions of kernel-tools 1 for target and 1 for SDKMACHINE however this at least brings back what we had. > Cheers, > > Richard > > >> Signed-off-by: Khem Raj <raj.khem@gmail.com> >> --- >> meta/classes/kernel.bbclass | 15 +++++++++++++-- >> 1 files changed, 13 insertions(+), 2 deletions(-) >> >> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass >> index 81c334b..b2b6fcd 100644 >> --- a/meta/classes/kernel.bbclass >> +++ b/meta/classes/kernel.bbclass >> @@ -80,6 +80,7 @@ KERNEL_ALT_IMAGETYPE ??= "" >> # Define where the kernel headers are installed on the target as well as where >> # they are staged. >> KERNEL_SRC_PATH = "/usr/src/kernel" >> +UNPRUNED_KERNEL_SRC_PATH = "/usr/src/kernel-unpruned" >> >> KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}" >> >> @@ -135,7 +136,9 @@ kernel_do_install() { >> # kernel source tree. >> # >> kerneldir=${D}${KERNEL_SRC_PATH} >> + unrprunedkerneldir=${D}${UNPRUNED_KERNEL_SRC_PATH} >> install -d $kerneldir >> + install -d $unrprunedkerneldir >> >> # >> # Store the kernel version in sysroots for module-base.bbclass >> @@ -176,7 +179,6 @@ kernel_do_install() { >> # and include files. >> # >> oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean >> - make -C $kerneldir _mrproper_scripts >> find $kerneldir -path $kerneldir/lib -prune -o -path $kerneldir/tools -prune -o -path $kerneldir/scripts -prune -o -name "*.[csS]" -exec rm '{}' \; >> find $kerneldir/Documentation -name "*.txt" -exec rm '{}' \; >> >> @@ -186,6 +188,9 @@ kernel_do_install() { >> if [ ${ARCH} = "powerpc" ]; then >> cp arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o >> fi >> + # make a copy of unpruned kernel tree >> + cp -fR $kerneldir/* $unrprunedkerneldir >> + make -C $kerneldir _mrproper_scripts >> >> # Remove the following binaries which cause strip or arch QA errors >> # during do_package for cross-compiled platforms >> @@ -197,8 +202,14 @@ kernel_do_install() { >> done >> } >> >> +PACKAGE_PREPROCESS_FUNCS += "kernel_package_preprocess" >> + >> +kernel_package_preprocess () { >> + rm -rf ${PKGD}/${UNPRUNED_KERNEL_SRC_PATH} >> +} >> + >> sysroot_stage_all_append() { >> - sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} >> + sysroot_stage_dir ${D}${UNPRUNED_KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} >> } >> >> kernel_do_configure() { > > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Wed, 2012-07-11 at 07:25 -0700, Khem Raj wrote: > On Wed, Jul 11, 2012 at 3:30 AM, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > On Tue, 2012-07-10 at 10:07 -0700, Khem Raj wrote: > >> We shave too much from kernel sources for making it work > >> for on device external kernel module development that cross > >> development of external modules wont work from same tree > >> anymore. This patch makes a copy of tree which will eventually > >> be staged for building external modules > > > > It sounds from the further discussion that there is more to the patch > > than just this. Firstly, a change like this needs a more precise > > description. > > > > Secondly, we're copying around *way* to much data in this approach. I'd > > like to suggest an improvement but can't since the description above is > > lacking, I can't even understand what the problem is you're trying to > > fix. > > alright. There are tools (especially in scripts/) which are made for > buildhost when compiling the kernel irrespective of cross > or native. The tools like fixdep, recordmcount etc. which are needed > to run when you build kernel itself as well as external > modules. Now we do 'make _mproper_scripts' which cleans all those > binaries. We do this because we want to ship kernel-dev > and packaging binaries for different host wont be allowed in a target > package. So we chose to delete them and then on device > regenerate them ( ideally I would have preferred to generate them > cross candian when making for target) > > Deleting these tools also renders the cross building of modules > useless. Since I want to ship the sources as reference only > meaning people may not have write access to the kernel sources they > can not run make inside the kernel sources to regenerate > those binaries before they start building their external modules. > > > > > So more discussion is needed. > > > > I have a suspicion that this fix only "happens" to work when SDKMACHINE > > == NATIVEMACHINE. > > > > yes, for a full solution we have to generate two versions of > kernel-tools 1 for target and 1 for SDKMACHINE > however this at least brings back what we had. s/full/non-broken/ This patch adds something that is broken in several different cases and kills off performance as an added bonus. I'd like to get this fixed properly please rather than perpetuate the problem. We need to either do something well and correctly, or not do it at all. We could document "make _mproper_scripts" as a requirement for installing the kernel SDK in the meantime. I think we do need to make a kernel-tools package which correctly generates the tools for a given target, be it nativesdk, or the target device. Cheers, Richard
On Tue, Jul 10, 2012 at 10:07:50AM -0700, Khem Raj wrote: > We shave too much from kernel sources for making it work > for on device external kernel module development that cross > development of external modules wont work from same tree > anymore. This patch makes a copy of tree which will eventually > be staged for building external modules Does this solve also this issue? http://lists.linuxtogo.org/pipermail/openembedded-devel/2012-July/040584.html Cheers, > Signed-off-by: Khem Raj <raj.khem@gmail.com> > --- > meta/classes/kernel.bbclass | 15 +++++++++++++-- > 1 files changed, 13 insertions(+), 2 deletions(-) > > diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass > index 81c334b..b2b6fcd 100644 > --- a/meta/classes/kernel.bbclass > +++ b/meta/classes/kernel.bbclass > @@ -80,6 +80,7 @@ KERNEL_ALT_IMAGETYPE ??= "" > # Define where the kernel headers are installed on the target as well as where > # they are staged. > KERNEL_SRC_PATH = "/usr/src/kernel" > +UNPRUNED_KERNEL_SRC_PATH = "/usr/src/kernel-unpruned" > > KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}" > > @@ -135,7 +136,9 @@ kernel_do_install() { > # kernel source tree. > # > kerneldir=${D}${KERNEL_SRC_PATH} > + unrprunedkerneldir=${D}${UNPRUNED_KERNEL_SRC_PATH} > install -d $kerneldir > + install -d $unrprunedkerneldir > > # > # Store the kernel version in sysroots for module-base.bbclass > @@ -176,7 +179,6 @@ kernel_do_install() { > # and include files. > # > oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean > - make -C $kerneldir _mrproper_scripts > find $kerneldir -path $kerneldir/lib -prune -o -path $kerneldir/tools -prune -o -path $kerneldir/scripts -prune -o -name "*.[csS]" -exec rm '{}' \; > find $kerneldir/Documentation -name "*.txt" -exec rm '{}' \; > > @@ -186,6 +188,9 @@ kernel_do_install() { > if [ ${ARCH} = "powerpc" ]; then > cp arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o > fi > + # make a copy of unpruned kernel tree > + cp -fR $kerneldir/* $unrprunedkerneldir > + make -C $kerneldir _mrproper_scripts > > # Remove the following binaries which cause strip or arch QA errors > # during do_package for cross-compiled platforms > @@ -197,8 +202,14 @@ kernel_do_install() { > done > } > > +PACKAGE_PREPROCESS_FUNCS += "kernel_package_preprocess" > + > +kernel_package_preprocess () { > + rm -rf ${PKGD}/${UNPRUNED_KERNEL_SRC_PATH} > +} > + > sysroot_stage_all_append() { > - sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} > + sysroot_stage_dir ${D}${UNPRUNED_KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} > } > > kernel_do_configure() { > -- > 1.7.5.4 > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On 07/11/2012 07:48 AM, Richard Purdie wrote: > On Wed, 2012-07-11 at 07:25 -0700, Khem Raj wrote: >> On Wed, Jul 11, 2012 at 3:30 AM, Richard Purdie >> <richard.purdie@linuxfoundation.org> wrote: >>> On Tue, 2012-07-10 at 10:07 -0700, Khem Raj wrote: >>>> We shave too much from kernel sources for making it work >>>> for on device external kernel module development that cross >>>> development of external modules wont work from same tree >>>> anymore. This patch makes a copy of tree which will eventually >>>> be staged for building external modules >>> >>> It sounds from the further discussion that there is more to the patch >>> than just this. Firstly, a change like this needs a more precise >>> description. >>> >>> Secondly, we're copying around *way* to much data in this approach. I'd >>> like to suggest an improvement but can't since the description above is >>> lacking, I can't even understand what the problem is you're trying to >>> fix. >> >> alright. There are tools (especially in scripts/) which are made for >> buildhost when compiling the kernel irrespective of cross >> or native. The tools like fixdep, recordmcount etc. which are needed >> to run when you build kernel itself as well as external >> modules. Now we do 'make _mproper_scripts' which cleans all those >> binaries. We do this because we want to ship kernel-dev >> and packaging binaries for different host wont be allowed in a target >> package. So we chose to delete them and then on device >> regenerate them ( ideally I would have preferred to generate them >> cross candian when making for target) >> >> Deleting these tools also renders the cross building of modules >> useless. Since I want to ship the sources as reference only >> meaning people may not have write access to the kernel sources they >> can not run make inside the kernel sources to regenerate >> those binaries before they start building their external modules. >> >>> >>> So more discussion is needed. >>> >>> I have a suspicion that this fix only "happens" to work when SDKMACHINE >>> == NATIVEMACHINE. >>> >> >> yes, for a full solution we have to generate two versions of >> kernel-tools 1 for target and 1 for SDKMACHINE >> however this at least brings back what we had. > > s/full/non-broken/ > > This patch adds something that is broken in several different cases and > kills off performance as an added bonus. I'd like to get this fixed > properly please rather than perpetuate the problem. We need to either do > something well and correctly, or not do it at all. We could document > "make _mproper_scripts" as a requirement for installing the kernel SDK > in the meantime. > > I think we do need to make a kernel-tools package which correctly > generates the tools for a given target, be it nativesdk, or the target > device. Khem, I take it we still have something left to do here in addition to the bounds.h patch you sent a short while ago? If so, this sounds like a big enough effort that a bug is warranted. Would you consider writing up exactly what you're trying to accomplish and opening a bug?
On Fri, Jul 20, 2012 at 12:30 AM, Darren Hart <dvhart@linux.intel.com> wrote: > Khem, I take it we still have something left to do here in addition to > the bounds.h patch you sent a short while ago? > > If so, this sounds like a big enough effort that a bug is warranted. > Would you consider writing up exactly what you're trying to accomplish > and opening a bug? my problem is bigger. I am trying to provide a (enough) prebuilt kernel tree so that external modules ( outside of build OE tree) can be built using SDK in cross environment but "make -C $kerneldir _mrproper_scripts" has bit me in the rear side since now it deletes considerable portion of kernel bits that the resulting tree is unusable for building without modifying the tree by running "make scripts" or so. bounds.h was for syncing with OE-Core, since I am in process of getting rid of copy of kernel.bbclass in meta-oe
Patch
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass index 81c334b..b2b6fcd 100644 --- a/meta/classes/kernel.bbclass +++ b/meta/classes/kernel.bbclass @@ -80,6 +80,7 @@ KERNEL_ALT_IMAGETYPE ??= "" # Define where the kernel headers are installed on the target as well as where # they are staged. KERNEL_SRC_PATH = "/usr/src/kernel" +UNPRUNED_KERNEL_SRC_PATH = "/usr/src/kernel-unpruned" KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz" else s)(d.getVar('KERNEL_IMAGETYPE', True))}" @@ -135,7 +136,9 @@ kernel_do_install() { # kernel source tree. # kerneldir=${D}${KERNEL_SRC_PATH} + unrprunedkerneldir=${D}${UNPRUNED_KERNEL_SRC_PATH} install -d $kerneldir + install -d $unrprunedkerneldir # # Store the kernel version in sysroots for module-base.bbclass @@ -176,7 +179,6 @@ kernel_do_install() { # and include files. # oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean - make -C $kerneldir _mrproper_scripts find $kerneldir -path $kerneldir/lib -prune -o -path $kerneldir/tools -prune -o -path $kerneldir/scripts -prune -o -name "*.[csS]" -exec rm '{}' \; find $kerneldir/Documentation -name "*.txt" -exec rm '{}' \; @@ -186,6 +188,9 @@ kernel_do_install() { if [ ${ARCH} = "powerpc" ]; then cp arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o fi + # make a copy of unpruned kernel tree + cp -fR $kerneldir/* $unrprunedkerneldir + make -C $kerneldir _mrproper_scripts # Remove the following binaries which cause strip or arch QA errors # during do_package for cross-compiled platforms @@ -197,8 +202,14 @@ kernel_do_install() { done } +PACKAGE_PREPROCESS_FUNCS += "kernel_package_preprocess" + +kernel_package_preprocess () { + rm -rf ${PKGD}/${UNPRUNED_KERNEL_SRC_PATH} +} + sysroot_stage_all_append() { - sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} + sysroot_stage_dir ${D}${UNPRUNED_KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} } kernel_do_configure() {
We shave too much from kernel sources for making it work for on device external kernel module development that cross development of external modules wont work from same tree anymore. This patch makes a copy of tree which will eventually be staged for building external modules Signed-off-by: Khem Raj <raj.khem@gmail.com> --- meta/classes/kernel.bbclass | 15 +++++++++++++-- 1 files changed, 13 insertions(+), 2 deletions(-)