| Submitter | Mark Hatle |
|---|---|
| Date | April 9, 2012, 3:17 p.m. |
| Message ID | <4F82FD70.9080609@windriver.com> |
| Download | mbox | patch |
| Permalink | /patch/25451/ |
| State | New |
| Headers | show |
Comments
Op 9 apr. 2012, om 17:17 heeft Mark Hatle het volgende geschreven: > On 4/8/12 4:34 PM, Andreas Oberritter wrote: >> On 07.04.2012 02:10, Mark Hatle wrote: >>> Just ran a local build with the qemumips machine, this is a standard >>> mips32 target. >>> >>> From the configure line for eglibc: >>> >>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure >>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux >>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin >>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc >>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib >>> --includedir=/usr/include --oldincludedir=/usr/include >>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules >>> --disable-dependency-tracking >>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips >>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug >>> --without-gd --enable-clocale=gnu >>> --enable-add-ons=ports,nptl,libidn,ports >>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include >>> --without-selinux >>> >>> The system is correctly setting the target to "mips-oe-linux". >>> >>> I checked and bash is the same way. >>> >>> So the canonical arch is correct, the mips32 is only the packaging >>> arch. It was always intended that the packaging arch be used in full on >>> MIPS. (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as >>> necessary if we expand the mips tunings.) >> >> I don't think such a change should be done only few days before a >> release. Until this patch was applied, the packaging arch has always >> been mipsel, not mips32el. Please, revert or fix this! > > This is easy to change to the previous behavior... however it was a bug in the original implementation. > > But again, I stress nothing changed except for the packaging arch... the way the packages are configured, compiled, installed remain the same, only the package arch has changed. "only"
On 4/9/12 10:56 AM, Koen Kooi wrote: > > Op 9 apr. 2012, om 17:17 heeft Mark Hatle het volgende geschreven: > >> On 4/8/12 4:34 PM, Andreas Oberritter wrote: >>> On 07.04.2012 02:10, Mark Hatle wrote: >>>> Just ran a local build with the qemumips machine, this is a standard >>>> mips32 target. >>>> >>>> From the configure line for eglibc: >>>> >>>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure >>>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux >>>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin >>>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc >>>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib >>>> --includedir=/usr/include --oldincludedir=/usr/include >>>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules >>>> --disable-dependency-tracking >>>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips >>>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug >>>> --without-gd --enable-clocale=gnu >>>> --enable-add-ons=ports,nptl,libidn,ports >>>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include >>>> --without-selinux >>>> >>>> The system is correctly setting the target to "mips-oe-linux". >>>> >>>> I checked and bash is the same way. >>>> >>>> So the canonical arch is correct, the mips32 is only the packaging >>>> arch. It was always intended that the packaging arch be used in full on >>>> MIPS. (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as >>>> necessary if we expand the mips tunings.) >>> >>> I don't think such a change should be done only few days before a >>> release. Until this patch was applied, the packaging arch has always >>> been mipsel, not mips32el. Please, revert or fix this! >> >> This is easy to change to the previous behavior... however it was a bug in the original implementation. >> >> But again, I stress nothing changed except for the packaging arch... the way the packages are configured, compiled, installed remain the same, only the package arch has changed. > > "only" Yes, only. The package arch is used by the feed system and the build of the images. I verified the images were still being generated corrected. I did not verify anything within external package feeds, as I have no way to easily do this. If anything else in the system is using the package arch as a key, then it's broken. The configure arch, the tuning, and similar are all reasonable things to use, but the package arch is arbitrary. We may have a fairly defined, defacto set, but they really are arbitrary and should not affect any software -- other then those directly working with the package feeds. --Mark > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On 09.04.2012 17:17, Mark Hatle wrote: > On 4/8/12 4:34 PM, Andreas Oberritter wrote: >> On 07.04.2012 02:10, Mark Hatle wrote: >>> Just ran a local build with the qemumips machine, this is a standard >>> mips32 target. >>> >>> From the configure line for eglibc: >>> >>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure >>> >>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux >>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin >>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc >>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib >>> --includedir=/usr/include --oldincludedir=/usr/include >>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules >>> --disable-dependency-tracking >>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips >>> >>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug >>> --without-gd --enable-clocale=gnu >>> --enable-add-ons=ports,nptl,libidn,ports >>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include >>> >>> --without-selinux >>> >>> The system is correctly setting the target to "mips-oe-linux". >>> >>> I checked and bash is the same way. >>> >>> So the canonical arch is correct, the mips32 is only the packaging >>> arch. It was always intended that the packaging arch be used in full on >>> MIPS. (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as >>> necessary if we expand the mips tunings.) >> >> I don't think such a change should be done only few days before a >> release. Until this patch was applied, the packaging arch has always >> been mipsel, not mips32el. Please, revert or fix this! > > This is easy to change to the previous behavior... however it was a bug > in the original implementation. > > But again, I stress nothing changed except for the packaging arch... the > way the packages are configured, compiled, installed remain the same, > only the package arch has changed. The only place that may be affected > by this is the package feed mechanism. I think breaking package feeds in such a way is one of the worst things to do in OE. > To revert to the previous behavior, in the > meta/conf/machine/include/tune-mips.inc: > > --- a/meta/conf/machine/include/tune-mips32.inc > +++ b/meta/conf/machine/include/tune-mips32.inc > @@ -9,17 +9,17 @@ TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", > "mips32" > AVAILTUNES += "mips32 mips32el mips32-nf mips32el-nf" > > TUNE_FEATURES_tune-mips32 = "${TUNE_FEATURES_tune-mips} mips32" > -MIPSPKGSFX_VARIANT_tune-mips32 = "mips32" > +MIPSPKGSFX_VARIANT_tune-mips32 = "mips" > PACKAGE_EXTRA_ARCHS_tune-mips32 = "mips mips32" > > TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32" > -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el" > +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel" > PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el" ^^^^^^^^ I don't think this is correct, in all four cases, because no packages will have that arch. > TUNE_FEATURES_tune-mips32-nf = "${TUNE_FEATURES_tune-mips-nf} mips32" > -MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips32" > +MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips" > PACKAGE_EXTRA_ARCHS_tune-mips32-nf = "mips-nf mips32-nf" > > TUNE_FEATURES_tune-mips32el-nf = "${TUNE_FEATURES_tune-mipsel-nf} mips32" > -MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mips32el" > +MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mipsel" > PACKAGE_EXTRA_ARCHS_tune-mips32el-nf = "mipsel-nf mips32el-nf" > > Before I submit this patch though, I would like others to weigh in on > the issue. This was a mistake in the earlier version of the code. The > "variant" wasn't be set as it should have been. > > The problem is that if you set the tune to "mips", you get the default > compiler behavior. According to the gcc docs, there is no "mips" ISA name. Valid names are: `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and `mips64r2'. Therefore I don't understand why "mips" should default to anything else, if it was an alias for mips32 before. > However, if you set the tune to mips32, you get > "-march=mips32" added to your CCARGS. This will produce slightly > different binaries, thus "mips" and mips32" are not equal. Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or mips32el, so this probably broke, too. meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still work, e.g. EXTRA_OECONF_mipsel etc.? How about meta/recipes-qt/qt4/qt4_arch.inc? Whatever the answers are, the most important point is that it's the worst point in time to do such a change. We should rather discuss it after the release, if at all. Regards, Andreas
On 4/9/12 3:06 PM, Andreas Oberritter wrote: > On 09.04.2012 17:17, Mark Hatle wrote: >> On 4/8/12 4:34 PM, Andreas Oberritter wrote: >>> On 07.04.2012 02:10, Mark Hatle wrote: >>>> Just ran a local build with the qemumips machine, this is a standard >>>> mips32 target. >>>> ... >>>> So the canonical arch is correct, the mips32 is only the packaging >>>> arch. It was always intended that the packaging arch be used in full on >>>> MIPS. (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as >>>> necessary if we expand the mips tunings.) >>> >>> I don't think such a change should be done only few days before a >>> release. Until this patch was applied, the packaging arch has always >>> been mipsel, not mips32el. Please, revert or fix this! >> >> This is easy to change to the previous behavior... however it was a bug >> in the original implementation. >> >> But again, I stress nothing changed except for the packaging arch... the >> way the packages are configured, compiled, installed remain the same, >> only the package arch has changed. The only place that may be affected >> by this is the package feed mechanism. > > I think breaking package feeds in such a way is one of the worst things > to do in OE. > >> To revert to the previous behavior, in the >> meta/conf/machine/include/tune-mips.inc: >> ... >> TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32" >> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el" >> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel" >> PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el" > ^^^^^^^^ > I don't think this is correct, in all four cases, because no packages > will have that arch. > ... >> Before I submit this patch though, I would like others to weigh in on >> the issue. This was a mistake in the earlier version of the code. The >> "variant" wasn't be set as it should have been. >> >> The problem is that if you set the tune to "mips", you get the default >> compiler behavior. > > According to the gcc docs, there is no "mips" ISA name. Valid names are: > `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and > `mips64r2'. Therefore I don't understand why "mips" should default to > anything else, if it was an alias for mips32 before. > We have two sets of available tunings: "mips" and "mips32" tunings.. (add el and -nf variants) These are -different- tunings and today the only way to notice the difference is based on the package arch. The package arch is NOT the target ISA. It's an arbitrary string "we" have come up with to let people know the architecture, ABI and optimizations used in producing specific software. "mips" indicates that it's using the default mips compiler options, whatever those may be. While mips32 says it is specifically tuned to the mips32 architecture settings. I honestly have no idea what the default compiler settings for mips are, but the point is the tunings are different. If you want the "MIPS" tune, you may not be able to run the items compiled with the -march=mips32 option. We have to have a way to reconcile this. >> However, if you set the tune to mips32, you get >> "-march=mips32" added to your CCARGS. This will produce slightly >> different binaries, thus "mips" and mips32" are not equal. > > Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or > mips32el, so this probably broke, too. > meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still > work, e.g. EXTRA_OECONF_mipsel etc.? How about > meta/recipes-qt/qt4/qt4_arch.inc? Overrides are on the GNU canonical arch (TUNE_ARCH) correct? If that is the case then "mips" or "mipsel" is the canonical arch. Again, we do NOT use the package arch for these settings! Below are the overrides and related elements from the bitbake.conf file: OVERRIDES = "${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:forcevariable" DISTROOVERRIDES ?= "${@d.getVar('DISTRO', True) or ''}" MACHINEOVERRIDES ?= "${MACHINE}" # Used by canadian-cross to handle string conversions on TARGET_ARCH where needed TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', True).replace("_", "-")}" TARGET_ARCH = "${TUNE_ARCH}" So my reading of this is that, unless overriden somewhere outside of bitbake.conf, the override does include the TUNE_ARCH, via the TARGET_ARCH, via the TRANSLATED_TARGET_ARCH. > > Whatever the answers are, the most important point is that it's the > worst point in time to do such a change. We should rather discuss it > after the release, if at all. In order to resolve this consider the following: We have two tunes, "mips" and "mips32", the difference being the -march=mips32 in the later case. In order to support both tunes, we need to have a way to differentiate between them on a package arch basis or we end up in a situation where we have two packages with different contents and no way to tell them apart. In order to reconcile the above, the three primary options are see are: *) Define only mips or mips32 tune, but not both -- producing "mips" as the package arch. (But then what do we do in the future about mips1, mips2, mips3, mips4, mips32r2?) *) Revert the behavior and have two tunes that produce the identical filename package with different contents and deal with this in the future. *) Keep it as it is now and produce mips and mips32 packages based on the specific tunings defined by the user We have a bug, I believe we need to fix it.. first or third options "fix" the bug.. the second option retains the bug to be fixed in the future. If you have an alternative to the above, I'm interested -- I just really don't like the "leave the bug" option. And just to be extra clear, I consider it a defect if we can produce a package with the same name for two different tune settings.. (the exception being the hell that is ARM and thumb namings.) --Mark > Regards, > Andreas > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On 09.04.2012 22:25, Mark Hatle wrote: > On 4/9/12 3:06 PM, Andreas Oberritter wrote: >> On 09.04.2012 17:17, Mark Hatle wrote: >>> On 4/8/12 4:34 PM, Andreas Oberritter wrote: >>>> On 07.04.2012 02:10, Mark Hatle wrote: >>>>> Just ran a local build with the qemumips machine, this is a standard >>>>> mips32 target. >>>>> > > ... > >>>>> So the canonical arch is correct, the mips32 is only the packaging >>>>> arch. It was always intended that the packaging arch be used in >>>>> full on >>>>> MIPS. (This will allow us to specify mips32r2, mipsiii, mipsiv, >>>>> etc as >>>>> necessary if we expand the mips tunings.) >>>> >>>> I don't think such a change should be done only few days before a >>>> release. Until this patch was applied, the packaging arch has always >>>> been mipsel, not mips32el. Please, revert or fix this! >>> >>> This is easy to change to the previous behavior... however it was a bug >>> in the original implementation. >>> >>> But again, I stress nothing changed except for the packaging arch... the >>> way the packages are configured, compiled, installed remain the same, >>> only the package arch has changed. The only place that may be affected >>> by this is the package feed mechanism. >> >> I think breaking package feeds in such a way is one of the worst things >> to do in OE. >> >>> To revert to the previous behavior, in the >>> meta/conf/machine/include/tune-mips.inc: >>> > > ... > >>> TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32" >>> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el" >>> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel" >>> PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el" >> ^^^^^^^^ >> I don't think this is correct, in all four cases, because no packages >> will have that arch. >> > > ... > >>> Before I submit this patch though, I would like others to weigh in on >>> the issue. This was a mistake in the earlier version of the code. The >>> "variant" wasn't be set as it should have been. >>> >>> The problem is that if you set the tune to "mips", you get the default >>> compiler behavior. >> >> According to the gcc docs, there is no "mips" ISA name. Valid names are: >> `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and >> `mips64r2'. Therefore I don't understand why "mips" should default to >> anything else, if it was an alias for mips32 before. >> > > We have two sets of available tunings: > > "mips" and "mips32" tunings.. (add el and -nf variants) > > These are -different- tunings and today the only way to notice the > difference is based on the package arch. The package arch is NOT the > target ISA. It's an arbitrary string "we" have come up with to let > people know the architecture, ABI and optimizations used in producing > specific software. "mips" indicates that it's using the default mips > compiler options, whatever those may be. While mips32 says it is > specifically tuned to the mips32 architecture settings. > > I honestly have no idea what the default compiler settings for mips are, > but the point is the tunings are different. If you want the "MIPS" > tune, you may not be able to run the items compiled with the > -march=mips32 option. We have to have a way to reconcile this. > >>> However, if you set the tune to mips32, you get >>> "-march=mips32" added to your CCARGS. This will produce slightly >>> different binaries, thus "mips" and mips32" are not equal. >> >> Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or >> mips32el, so this probably broke, too. >> meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still >> work, e.g. EXTRA_OECONF_mipsel etc.? How about >> meta/recipes-qt/qt4/qt4_arch.inc? > > Overrides are on the GNU canonical arch (TUNE_ARCH) correct? If that is > the case then "mips" or "mipsel" is the canonical arch. Again, we do > NOT use the package arch for these settings! > > Below are the overrides and related elements from the bitbake.conf file: > > OVERRIDES = > "${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:forcevariable" > > DISTROOVERRIDES ?= "${@d.getVar('DISTRO', True) or ''}" > MACHINEOVERRIDES ?= "${MACHINE}" > > # Used by canadian-cross to handle string conversions on TARGET_ARCH > where needed > TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', > True).replace("_", "-")}" > > TARGET_ARCH = "${TUNE_ARCH}" > > So my reading of this is that, unless overriden somewhere outside of > bitbake.conf, the override does include the TUNE_ARCH, via the > TARGET_ARCH, via the TRANSLATED_TARGET_ARCH. > >> >> Whatever the answers are, the most important point is that it's the >> worst point in time to do such a change. We should rather discuss it >> after the release, if at all. > > In order to resolve this consider the following: > > We have two tunes, "mips" and "mips32", the difference being the > -march=mips32 in the later case. > > In order to support both tunes, we need to have a way to differentiate > between them on a package arch basis or we end up in a situation where > we have two packages with different contents and no way to tell them apart. > > In order to reconcile the above, the three primary options are see are: > > *) Define only mips or mips32 tune, but not both -- producing "mips" as > the package arch. (But then what do we do in the future about mips1, > mips2, mips3, mips4, mips32r2?) > > *) Revert the behavior and have two tunes that produce the identical > filename package with different contents and deal with this in the future. > > *) Keep it as it is now and produce mips and mips32 packages based on > the specific tunings defined by the user > > We have a bug, I believe we need to fix it.. first or third options > "fix" the bug.. the second option retains the bug to be fixed in the > future. > > If you have an alternative to the above, I'm interested -- I just really > don't like the "leave the bug" option. > > And just to be extra clear, I consider it a defect if we can produce a > package with the same name for two different tune settings.. (the > exception being the hell that is ARM and thumb namings.) qemumips uses "mips32", so let's drop the "mips" tune for now and defer any other changes until after the release.
On 4/9/12 3:25 PM, Mark Hatle wrote: > On 4/9/12 3:06 PM, Andreas Oberritter wrote: >> On 09.04.2012 17:17, Mark Hatle wrote: >>> On 4/8/12 4:34 PM, Andreas Oberritter wrote: >>>> On 07.04.2012 02:10, Mark Hatle wrote: >>>>> Just ran a local build with the qemumips machine, this is a standard >>>>> mips32 target. >>>>> > > ... > >>>>> So the canonical arch is correct, the mips32 is only the packaging >>>>> arch. It was always intended that the packaging arch be used in full on >>>>> MIPS. (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as >>>>> necessary if we expand the mips tunings.) >>>> >>>> I don't think such a change should be done only few days before a >>>> release. Until this patch was applied, the packaging arch has always >>>> been mipsel, not mips32el. Please, revert or fix this! >>> >>> This is easy to change to the previous behavior... however it was a bug >>> in the original implementation. >>> >>> But again, I stress nothing changed except for the packaging arch... the >>> way the packages are configured, compiled, installed remain the same, >>> only the package arch has changed. The only place that may be affected >>> by this is the package feed mechanism. >> >> I think breaking package feeds in such a way is one of the worst things >> to do in OE. >> >>> To revert to the previous behavior, in the >>> meta/conf/machine/include/tune-mips.inc: >>> > > ... > >>> TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32" >>> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el" >>> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel" >>> PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el" >> ^^^^^^^^ >> I don't think this is correct, in all four cases, because no packages >> will have that arch. >> > > ... > >>> Before I submit this patch though, I would like others to weigh in on >>> the issue. This was a mistake in the earlier version of the code. The >>> "variant" wasn't be set as it should have been. >>> >>> The problem is that if you set the tune to "mips", you get the default >>> compiler behavior. >> >> According to the gcc docs, there is no "mips" ISA name. Valid names are: >> `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and >> `mips64r2'. Therefore I don't understand why "mips" should default to >> anything else, if it was an alias for mips32 before. >> > > We have two sets of available tunings: > > "mips" and "mips32" tunings.. (add el and -nf variants) > > These are -different- tunings and today the only way to notice the difference is > based on the package arch. The package arch is NOT the target ISA. It's an > arbitrary string "we" have come up with to let people know the architecture, ABI > and optimizations used in producing specific software. "mips" indicates that > it's using the default mips compiler options, whatever those may be. While > mips32 says it is specifically tuned to the mips32 architecture settings. > > I honestly have no idea what the default compiler settings for mips are, but the > point is the tunings are different. If you want the "MIPS" tune, you may not be > able to run the items compiled with the -march=mips32 option. We have to have a > way to reconcile this. I did some further digging into the GCC configuration. The default configuration is defined in the various defines: #define _R3000 1 #define __mips_fpr 32 #define _MIPS_ARCH_MIPS1 1 #define _MIPS_ARCH "mips1" #define _MIPS_TUNE_MIPS1 1 #define _MIPS_TUNE "mips1" #define __mips 1 #define _MIPS_ISA _MIPS_ISA_MIPS1 The default is defined for the MIPS1 architecture. The -march=mips32 changes the above to: #define __mips__ 1 #define _mips 1 #define mips 1 #define __R3000 1 #define __R3000__ 1 #define R3000 1 #define _R3000 1 #define __mips_fpr 32 #define _MIPS_ARCH_MIPS32 1 #define _MIPS_ARCH "mips32" #define _MIPS_TUNE_MIPS32 1 #define _MIPS_TUNE "mips32" #define __mips 32 #define __mips_isa_rev 1 #define _MIPS_ISA _MIPS_ISA_MIPS32 Internally in gcc the different between the two is processor optimizations change from the R3000 to the MIPS 4Kc, and PTF_AVOID_BRANCHLIKELY is defined. PTF_AVOID_BRANCHLIKELY Set if it is usually not profitable to use branch-likely instructions for this target, typically because the branches are always predicted taken and so incur a large overhead when not taken. So there will in fact be a difference in the generated binaries. >>> However, if you set the tune to mips32, you get >>> "-march=mips32" added to your CCARGS. This will produce slightly >>> different binaries, thus "mips" and mips32" are not equal. >> >> Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or >> mips32el, so this probably broke, too. >> meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still >> work, e.g. EXTRA_OECONF_mipsel etc.? How about >> meta/recipes-qt/qt4/qt4_arch.inc? > > Overrides are on the GNU canonical arch (TUNE_ARCH) correct? If that is the > case then "mips" or "mipsel" is the canonical arch. Again, we do NOT use the > package arch for these settings! > > Below are the overrides and related elements from the bitbake.conf file: > > OVERRIDES = > "${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:forcevariable" > DISTROOVERRIDES ?= "${@d.getVar('DISTRO', True) or ''}" > MACHINEOVERRIDES ?= "${MACHINE}" > > # Used by canadian-cross to handle string conversions on TARGET_ARCH where needed > TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', True).replace("_", "-")}" > > TARGET_ARCH = "${TUNE_ARCH}" > > So my reading of this is that, unless overriden somewhere outside of > bitbake.conf, the override does include the TUNE_ARCH, via the TARGET_ARCH, via > the TRANSLATED_TARGET_ARCH. > >> >> Whatever the answers are, the most important point is that it's the >> worst point in time to do such a change. We should rather discuss it >> after the release, if at all. > > In order to resolve this consider the following: > > We have two tunes, "mips" and "mips32", the difference being the -march=mips32 > in the later case. > > In order to support both tunes, we need to have a way to differentiate between > them on a package arch basis or we end up in a situation where we have two > packages with different contents and no way to tell them apart. > > In order to reconcile the above, the three primary options are see are: > > *) Define only mips or mips32 tune, but not both -- producing "mips" as the > package arch. (But then what do we do in the future about mips1, mips2, mips3, > mips4, mips32r2?) > > *) Revert the behavior and have two tunes that produce the identical filename > package with different contents and deal with this in the future. > > *) Keep it as it is now and produce mips and mips32 packages based on the > specific tunings defined by the user > > We have a bug, I believe we need to fix it.. first or third options "fix" the > bug.. the second option retains the bug to be fixed in the future. > > If you have an alternative to the above, I'm interested -- I just really don't > like the "leave the bug" option. > > And just to be extra clear, I consider it a defect if we can produce a package > with the same name for two different tune settings.. (the exception being the > hell that is ARM and thumb namings.) > > --Mark > >> Regards, >> Andreas >> >> _______________________________________________ >> Openembedded-core mailing list >> Openembedded-core@lists.openembedded.org >> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Mon, 2012-04-09 at 15:25 -0500, Mark Hatle wrote: > And just to be extra clear, I consider it a defect if we can produce a package > with the same name for two different tune settings.. (the exception being the > hell that is ARM and thumb namings.) While you might consider that a defect (and it probably is a defensible position to do so), it hasn't historically been considered such in OE. The PACKAGE_ARCH value has, traditionally, been concerned purely with ISA and ABI (i.e. answering the question "can I execute this code?") rather than optimisations. For example, the tune-arm926ejs.inc and tune-xscale.inc files in current oe-core both end up setting PACKAGE_ARCH to "armv5tte" (sic). But those are quite different processors and have different tuning requirements, so the binaries you get are unlikely to be the same. If you were to take the view that the PACKAGE_ARCH must uniquely identify one set of binaries then obviously each of these tunings (and probably all the ARM cpu-specific tunings) would need to set PACKAGE_ARCH to some unique value. p.
On 4/9/12 4:03 PM, Phil Blundell wrote: > On Mon, 2012-04-09 at 15:25 -0500, Mark Hatle wrote: >> And just to be extra clear, I consider it a defect if we can produce a package >> with the same name for two different tune settings.. (the exception being the >> hell that is ARM and thumb namings.) > > While you might consider that a defect (and it probably is a defensible > position to do so), it hasn't historically been considered such in OE. > The PACKAGE_ARCH value has, traditionally, been concerned purely with > ISA and ABI (i.e. answering the question "can I execute this code?") > rather than optimisations. > > For example, the tune-arm926ejs.inc and tune-xscale.inc files in current > oe-core both end up setting PACKAGE_ARCH to "armv5tte" (sic). But those > are quite different processors and have different tuning requirements, > so the binaries you get are unlikely to be the same. If you were to > take the view that the PACKAGE_ARCH must uniquely identify one set of > binaries then obviously each of these tunings (and probably all the ARM > cpu-specific tunings) would need to set PACKAGE_ARCH to some unique > value. I do, and thus the hell that is ARM. I could not currently generate a single package feed that work would on a variety of devices (like a traditional workstaton/server Linux OS would.) While this isn't a big issue in the embedded space where we should hopefully be aware of the tunings and configuration were are using, it is still a problem. As the systems get larger, the requirement for common pages feeds increases, leading to the problem being, well a problem. (ARM is starting to be considered for Carrier Grade systems, many of which have very common requirements to traditional server Linux. A set of established binaries and the vendors just want to drop in optimized applications.) On ARM what we currently have defined is: (tune) - (package arch) arm1136jfs - armv6-vfp arm920t - armv4t arm926ejs - armv5te arm9tdmi - armv4t armv4b - armv4b armv4tb - armv4tb armv4 - armv4 armv4t - armv4t armv5b - armv5b armv5b-vfp - armv5b-vfp armv5eb - armv5eb armv5eb-vfp - armv5eb-vfp armv5ehfb-vfp - armv5ehfb-vfp armv5ehf-vfp - armv5ehf-vfp armv5e-vfp - armv5e-vfp armv5hfb-vfp - armv5hfb-vfp armv5hf-vfp - armv5hf-vfp armv5tb - armv5tb armv5tb-vfp - armv5b-vfp armv5teb - armv5teb armv5teb-vfp - armv5teb-vfp armv5tehfb-vfp - armv5tehfb-vfp armv5tehf-vfp - armv5tehf-vfp armv5te - armv5te armv5te-vfp - armv5te-vfp armv5thfb-vfp - armv5hfb-vfp armv5thf-vfp - armv5hf-vfp armv5 - armv5 armv5t - armv5t armv5t-vfp - armv5-vfp armv5-vfp - armv5-vfp armv6b - armv6b-vfp armv6hfb - armv6hfb-vfp armv6hf - armv6hf-vfp armv6tb - armv6tb-vfp armv6thfb - armv6thfb-vfp armv6thf - armv6thf-vfp armv6 - armv6-vfp armv6t - armv6t-vfp armv7ab-neon - armv7ab-vfp-neon armv7ab - armv7ab-vfp armv7ahfb-neon - armv7ahfb-vfp-neon armv7ahfb - armv7ahfb-vfp armv7ahf-neon - armv7ahf-vfp-neon armv7ahf - armv7ahf-vfp armv7a-neon - armv7a-vfp-neon armv7atb-neon - armv7ab-vfp-neon armv7atb - armv7ab-vfp armv7athfb - armv7ahfb-vfp armv7athf-neon - armv7ahf-vfp-neon armv7athf - armv7ahf-vfp armv7a - armv7a-vfp armv7at-neon - armv7a-vfp-neon armv7at - armv7a-vfp cortexa8hf-neon - armv7ahf-vfp-neon cortexa8hf - armv7ahf-vfp cortexa8-neon - armv7a-vfp-neon cortexa8thf - armv7ahf-vfp cortexa8 - armv7a-vfp cortexa8t - armv7a-vfp cortexa9hf-neon - armv7ahf-vfp-neon cortexa9hf - armv7ahf-vfp cortexa9-neon - armv7a-vfp-neon cortexa9thf - armv7ahf-vfp cortexa9 - armv7a-vfp cortexa9t - armv7a-vfp cortexm1 - armv7a-vfp cortexm3 - armv7m-vfp cortexr4 - armv7r-vfp ep9312 - ep9312 iwmmxt - iwmmxt strongarm - armv4 Add in to that one of the tunings -- not indicated by the package arch of thumb enabled or not, and its difficult to know exactly what is in a package without extracting and examining it. But I consider this to be a quirk of the ARM architecture as implemented in OE. --Mark > p. > > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Mon, 2012-04-09 at 16:21 -0500, Mark Hatle wrote: > I do, and thus the hell that is ARM. I could not currently generate a single > package feed that work would on a variety of devices (like a traditional > workstaton/server Linux OS would.) Well, actually, you could in fact do exactly that. What you couldn't necessarily do with the tunings as they exist right now is generate a package feed which is optimised for (as opposed to "works on") all those devices. But it isn't clear to me that you could do that with a "traditional workstaton/server" kind of distribution either. In the x86 world, for example, the majority of the big distros do not bother to ship individually-tuned binaries for different processor types, certainly not for the entire distribution. >Add in to that one of the tunings -- not indicated by the package arch >of thumb enabled or not There are multiple reasons why this isn't indicated by the PACKAGE_ARCH. Firstly, it's irrelevant: on v5T or newer, the question of whether a given package is using Thumb-state or not has no ABI impact and there is no reason for anyone to care at a compatibility level. Second, it may be unpredictable: the compiler is at liberty (although current versions of gcc don't exploit this latitude) to switch arbitrarily between ARM-state and Thumb-state as it sees fit to get the best performance. And thirdly, it's just another piece of distro policy in the same way as compiling for -O2 vs -Os (which we also don't encode into PACKAGE_ARCH) is. p.
On 4/9/12 4:30 PM, Phil Blundell wrote: > On Mon, 2012-04-09 at 16:21 -0500, Mark Hatle wrote: >> I do, and thus the hell that is ARM. I could not currently generate a single >> package feed that work would on a variety of devices (like a traditional >> workstaton/server Linux OS would.) > > Well, actually, you could in fact do exactly that. What you couldn't > necessarily do with the tunings as they exist right now is generate a > package feed which is optimised for (as opposed to "works on") all those > devices. But it isn't clear to me that you could do that with a > "traditional workstaton/server" kind of distribution either. In the x86 > world, for example, the majority of the big distros do not bother to > ship individually-tuned binaries for different processor types, > certainly not for the entire distribution. Depends on the distribution and reasons for these feeds. What is typical is that a base distribution will be generated for a common compatible (reasonable) architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, etc) for the target arch, i.e. armv7a. Then you have a couple of packages hand-tuned for size, speed, or other that define or not thumb and add even a higher level of optimization. It's possible, folks do it today.. but it's not always obvious. (I have existing customers today that run a mix like I described through their own package feed like system. They really don't care at all that the core system is tuned for a given processor -- they only care that their specific applications and certain areas are specifically tuned to their use-cases.) Note, this is not what I would consider a typical use-case! >> Add in to that one of the tunings -- not indicated by the package arch >> of thumb enabled or not > > There are multiple reasons why this isn't indicated by the PACKAGE_ARCH. > Firstly, it's irrelevant: on v5T or newer, the question of whether a > given package is using Thumb-state or not has no ABI impact and there is > no reason for anyone to care at a compatibility level. Second, it may > be unpredictable: the compiler is at liberty (although current versions > of gcc don't exploit this latitude) to switch arbitrarily between > ARM-state and Thumb-state as it sees fit to get the best performance. > And thirdly, it's just another piece of distro policy in the same way as > compiling for -O2 vs -Os (which we also don't encode into PACKAGE_ARCH) > is. I agree, on ARM the tunings and optimizations between regular and thumb do not impact the ABI what-so-ever. And so far compilers have to be explicitly set to do thumb or tranditional ARM mode.. so in the end developers are looking into the performance and size impacts of each of these configuration and making changes as they see fit to best meet their needs. These are unique cases though, the majority of the software built for the core OS uses a single policy -- it's when something needs to be further optimized that this comes into play. At this point, I'd "like" to better differentiate the ARM package arches.. I don't care so much about the thumb enabled or not.. but the other tune settings are things I do care about. I started to change that for the last update and decided it was a rat-hole I was not willing to go down at this point. At some point in the future, I will look at, and document the differences in the tunings according to GCC configurations -- to get a good idea of what is and isn't producing the same binaries based on various arch and tune settings. --Mark > p. > > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Mon, Apr 9, 2012 at 1:25 PM, Mark Hatle <mark.hatle@windriver.com> wrote: > > In order to reconcile the above, the three primary options are see are: > > *) Define only mips or mips32 tune, but not both -- producing "mips" as the > package arch. (But then what do we do in the future about mips1, mips2, > mips3, mips4, mips32r2?) > > *) Revert the behavior and have two tunes that produce the identical > filename package with different contents and deal with this in the future. > > *) Keep it as it is now and produce mips and mips32 packages based on the > specific tunings defined by the user I think situation is not something I like but I would think that 1st option is better. Any machine that has been using mips32 tuning accidently could now use mips tune files and get same PKG_ARCH and we move mips32 tune into mips tune and hereon say that mips 32bit in OE defaults to -march=mips32 we could also change compiler defaults to use -march=mips32 to match the baseconfig and since qemumips has been using mips32 anyway that sort of is base mips arch. > > We have a bug, I believe we need to fix it.. first or third options "fix" > the bug.. the second option retains the bug to be fixed in the future. > > If you have an alternative to the above, I'm interested -- I just really > don't like the "leave the bug" option. > > And just to be extra clear, I consider it a defect if we can produce a > package with the same name for two different tune settings.. (the exception > being the hell that is ARM and thumb namings.)
On Mon, 2012-04-09 at 16:44 -0500, Mark Hatle wrote: > Depends on the distribution and reasons for these feeds. What is typical is > that a base distribution will be generated for a common compatible (reasonable) > architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, > etc) for the target arch, i.e. armv7a. Then you have a couple of packages > hand-tuned for size, speed, or other that define or not thumb and add even a > higher level of optimization. It's possible, folks do it today.. but it's not > always obvious. (I have existing customers today that run a mix like I > described through their own package feed like system. They really don't care at > all that the core system is tuned for a given processor -- they only care that > their specific applications and certain areas are specifically tuned to their > use-cases.) Note, this is not what I would consider a typical use-case! Sorry, I'm not quite sure I understand what point you're trying to make here. Are you describing what your customers are currently doing with OE, or are you saying that this is something that they would like to do with OE but don't feel they are able to at present, or something else? I'm still not entirely clear on what you feel is broken about the current state of the ARM tunings. What exactly is the scenario that works with the "traditional workstaton/server Linux OS" and can't be replicated with OE? But, all that aside, it seems ultimately that the exact way the PACKAGE_ARCHs are structured ought to be a DISTRO policy decision and not something that's mandated by the underlying infrastructure. That would perhaps remove some of the need for tinkering with these things in oe-core itself. p.
On 4/10/12 4:23 AM, Phil Blundell wrote: > On Mon, 2012-04-09 at 16:44 -0500, Mark Hatle wrote: >> Depends on the distribution and reasons for these feeds. What is typical is >> that a base distribution will be generated for a common compatible (reasonable) >> architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, >> etc) for the target arch, i.e. armv7a. Then you have a couple of packages >> hand-tuned for size, speed, or other that define or not thumb and add even a >> higher level of optimization. It's possible, folks do it today.. but it's not >> always obvious. (I have existing customers today that run a mix like I >> described through their own package feed like system. They really don't care at >> all that the core system is tuned for a given processor -- they only care that >> their specific applications and certain areas are specifically tuned to their >> use-cases.) Note, this is not what I would consider a typical use-case! > Sorry, I'm not quite sure I understand what point you're trying to make > here. Are you describing what your customers are currently doing with > OE, or are you saying that this is something that they would like to do > with OE but don't feel they are able to at present, or something else? The company I work for has an existing product that does not use OE. The customers using this have requested from us packages tagged with different package architectures to indicate the tuning and optimization information so that they can create multiple boards with the same software running on them. (This is closer to the traditional workstation/server model in my experience.) The multiple boards have a common set of OS applications that run on them. ARMv5 for the most part. The customers then have optimized applications with or without thumb, and optimized for a variety of different ARM parts. They then use the binary packages to assemble the filesystems, and perform field upgrades, on these boards as they are put into use. The installation system uses a best to least best match when doing assembly actions. So if the part is an ARMv7a, it will first look for ARMv7a w/ thumb, vfp and neon, not finding that, ARMv7a w/ thumb and vfp, then ARMv7a w/ thumb, then ARMv5 w/ thumb and vfp, then ARMv5 w/ thumb, and finally fall back to the ARMv5 binaries. > I'm still not entirely clear on what you feel is broken about the > current state of the ARM tunings. What exactly is the scenario that > works with the "traditional workstaton/server Linux OS" and can't be > replicated with OE? With the current implementation, there is no package differentiation between thumb and non-thumb binaries. I accept why this is in OE-core and I can live with that. However, there are other binaries that are theoretically optimized from specific Cortex models to the more generic ARMv7a tunings. Currently they all use the same package arch, which means I can't tell which CPU they're really for -- and this model (above) of best to least best match doesn't work. On a pure embedded model, I doubt anyone would do this. Thus it is a fairly unique embedded use-case, but a common Workstation/Server use case that is being replicated. > But, all that aside, it seems ultimately that the exact way the > PACKAGE_ARCHs are structured ought to be a DISTRO policy decision and > not something that's mandated by the underlying infrastructure. That > would perhaps remove some of the need for tinkering with these things in > oe-core itself. I intend, after this release, to propose changes to differentiate the models in oe-core. If the oe-core folks do not feel this is necessary, they I will maintain them on my own as I feel necessary to cover the above use-case. I can very much understand that in OE, for ARM specifically the package arch is simply indicating basic compatibility and not ABI & ISA & Optimization like it is on other architectures. --Mark > p. > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Tue, 2012-04-10 at 12:39 -0500, Mark Hatle wrote: > The installation system uses a best > to least best match when doing assembly actions. So if the part is an ARMv7a, > it will first look for ARMv7a w/ thumb, vfp and neon, not finding that, ARMv7a > w/ thumb and vfp, then ARMv7a w/ thumb, then ARMv5 w/ thumb and vfp, then ARMv5 > w/ thumb, and finally fall back to the ARMv5 binaries. This sounds like exactly the behaviour you would get with the current ARM tunings (except the Thumb bit which, as previously discussed, I think is somewhat misguided in the first place). The existing ARM tunings do seem to correctly encode VFP and Neon-ness. Which part isn't working for you? Maybe you could give a concrete example of where exactly it falls down. > I can very much understand that in OE, for ARM specifically the package arch is > simply indicating basic compatibility and not ABI & ISA & Optimization like it > is on other architectures. Well, I would consider "ABI & ISA" to be a fairly big part of "basic compatibility". It is true that we don't currently encode optimisations, but as I previously mentioned I don't think many (perhaps any) other distributions do that either, and it's perhaps debatable whether it would be a very useful thing to do in the general case. For individual packages you can obviously force the issue in your DISTRO configuration anyway. p.
Patch
--- a/meta/conf/machine/include/tune-mips32.inc +++ b/meta/conf/machine/include/tune-mips32.inc @@ -9,17 +9,17 @@ TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "mips32" AVAILTUNES += "mips32 mips32el mips32-nf mips32el-nf" TUNE_FEATURES_tune-mips32 = "${TUNE_FEATURES_tune-mips} mips32" -MIPSPKGSFX_VARIANT_tune-mips32 = "mips32" +MIPSPKGSFX_VARIANT_tune-mips32 = "mips" PACKAGE_EXTRA_ARCHS_tune-mips32 = "mips mips32" TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32" -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el" +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel" PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el" TUNE_FEATURES_tune-mips32-nf = "${TUNE_FEATURES_tune-mips-nf} mips32" -MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips32" +MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips" PACKAGE_EXTRA_ARCHS_tune-mips32-nf = "mips-nf mips32-nf" TUNE_FEATURES_tune-mips32el-nf = "${TUNE_FEATURES_tune-mipsel-nf} mips32" -MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mips32el" +MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mipsel" PACKAGE_EXTRA_ARCHS_tune-mips32el-nf = "mipsel-nf mips32el-nf"