Patchwork [3/7] conf/machine/include: Cleanup MIPS tunings to match README

login
register
mail settings
Submitter Mark Hatle
Date April 9, 2012, 3:17 p.m.
Message ID <4F82FD70.9080609@windriver.com>
Download mbox | patch
Permalink /patch/25451/
State New
Headers show

Comments

Mark Hatle - April 9, 2012, 3:17 p.m.
On 4/8/12 4:34 PM, Andreas Oberritter wrote:
> On 07.04.2012 02:10, Mark Hatle wrote:
>> Just ran a local build with the qemumips machine, this is a standard
>> mips32 target.
>>
>>  From the configure line for eglibc:
>>
>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure
>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux
>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc
>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib
>> --includedir=/usr/include --oldincludedir=/usr/include
>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules
>> --disable-dependency-tracking
>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips
>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug
>> --without-gd --enable-clocale=gnu
>> --enable-add-ons=ports,nptl,libidn,ports
>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include
>> --without-selinux
>>
>> The system is correctly setting the target to "mips-oe-linux".
>>
>> I checked and bash is the same way.
>>
>> So the canonical arch is correct, the mips32 is only the packaging
>> arch.  It was always intended that the packaging arch be used in full on
>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as
>> necessary if we expand the mips tunings.)
>
> I don't think such a change should be done only few days before a
> release. Until this patch was applied, the packaging arch has always
> been mipsel, not mips32el. Please, revert or fix this!

This is easy to change to the previous behavior...  however it was a bug in the 
original implementation.

But again, I stress nothing changed except for the packaging arch... the way the 
packages are configured, compiled, installed remain the same, only the package 
arch has changed.  The only place that may be affected by this is the package 
feed mechanism.

To revert to the previous behavior, in the meta/conf/machine/include/tune-mips.inc:


Before I submit this patch though, I would like others to weigh in on the issue. 
  This was a mistake in the earlier version of the code.  The "variant" wasn't 
be set as it should have been.

The problem is that if you set the tune to "mips", you get the default compiler 
behavior.  However, if you set the tune to mips32, you get "-march=mips32" added 
to your CCARGS.  This will produce slightly different binaries, thus "mips" and 
mips32" are not equal.

--Mark

>> So right now, I don't see any failure conditions with an oe-core build.
>> (This is oe-core as of earlier today.)
>>
>> --Mark
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Koen Kooi - April 9, 2012, 3:56 p.m.
Op 9 apr. 2012, om 17:17 heeft Mark Hatle het volgende geschreven:

> On 4/8/12 4:34 PM, Andreas Oberritter wrote:
>> On 07.04.2012 02:10, Mark Hatle wrote:
>>> Just ran a local build with the qemumips machine, this is a standard
>>> mips32 target.
>>> 
>>> From the configure line for eglibc:
>>> 
>>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure
>>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux
>>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
>>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc
>>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib
>>> --includedir=/usr/include --oldincludedir=/usr/include
>>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules
>>> --disable-dependency-tracking
>>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips
>>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug
>>> --without-gd --enable-clocale=gnu
>>> --enable-add-ons=ports,nptl,libidn,ports
>>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include
>>> --without-selinux
>>> 
>>> The system is correctly setting the target to "mips-oe-linux".
>>> 
>>> I checked and bash is the same way.
>>> 
>>> So the canonical arch is correct, the mips32 is only the packaging
>>> arch.  It was always intended that the packaging arch be used in full on
>>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as
>>> necessary if we expand the mips tunings.)
>> 
>> I don't think such a change should be done only few days before a
>> release. Until this patch was applied, the packaging arch has always
>> been mipsel, not mips32el. Please, revert or fix this!
> 
> This is easy to change to the previous behavior...  however it was a bug in the original implementation.
> 
> But again, I stress nothing changed except for the packaging arch... the way the packages are configured, compiled, installed remain the same, only the package arch has changed. 

"only"
Mark Hatle - April 9, 2012, 4:03 p.m.
On 4/9/12 10:56 AM, Koen Kooi wrote:
>
> Op 9 apr. 2012, om 17:17 heeft Mark Hatle het volgende geschreven:
>
>> On 4/8/12 4:34 PM, Andreas Oberritter wrote:
>>> On 07.04.2012 02:10, Mark Hatle wrote:
>>>> Just ran a local build with the qemumips machine, this is a standard
>>>> mips32 target.
>>>>
>>>>  From the configure line for eglibc:
>>>>
>>>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure
>>>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux
>>>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
>>>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc
>>>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib
>>>> --includedir=/usr/include --oldincludedir=/usr/include
>>>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules
>>>> --disable-dependency-tracking
>>>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips
>>>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug
>>>> --without-gd --enable-clocale=gnu
>>>> --enable-add-ons=ports,nptl,libidn,ports
>>>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include
>>>> --without-selinux
>>>>
>>>> The system is correctly setting the target to "mips-oe-linux".
>>>>
>>>> I checked and bash is the same way.
>>>>
>>>> So the canonical arch is correct, the mips32 is only the packaging
>>>> arch.  It was always intended that the packaging arch be used in full on
>>>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as
>>>> necessary if we expand the mips tunings.)
>>>
>>> I don't think such a change should be done only few days before a
>>> release. Until this patch was applied, the packaging arch has always
>>> been mipsel, not mips32el. Please, revert or fix this!
>>
>> This is easy to change to the previous behavior...  however it was a bug in the original implementation.
>>
>> But again, I stress nothing changed except for the packaging arch... the way the packages are configured, compiled, installed remain the same, only the package arch has changed.
>
> "only"

Yes, only.  The package arch is used by the feed system and the build of the 
images.  I verified the images were still being generated corrected.  I did not 
verify anything within external package feeds, as I have no way to easily do this.

If anything else in the system is using the package arch as a key, then it's 
broken.  The configure arch, the tuning, and similar are all reasonable things 
to use, but the package arch is arbitrary.  We may have a fairly defined, 
defacto set, but they really are arbitrary and should not affect any software -- 
other then those directly working with the package feeds.

--Mark

>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Andreas Oberritter - April 9, 2012, 8:06 p.m.
On 09.04.2012 17:17, Mark Hatle wrote:
> On 4/8/12 4:34 PM, Andreas Oberritter wrote:
>> On 07.04.2012 02:10, Mark Hatle wrote:
>>> Just ran a local build with the qemumips machine, this is a standard
>>> mips32 target.
>>>
>>>  From the configure line for eglibc:
>>>
>>> /msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/work/mips32-oe-linux/eglibc-2.13-r23+svnr15508/eglibc-2_13/libc/configure
>>>
>>> --build=x86_64-linux --host=mips-oe-linux --target=mips-oe-linux
>>> --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
>>> --libexecdir=/usr/libexec --datadir=/usr/share --sysconfdir=/etc
>>> --sharedstatedir=/com --localstatedir=/var --libdir=/usr/lib
>>> --includedir=/usr/include --oldincludedir=/usr/include
>>> --infodir=/usr/share/info --mandir=/usr/share/man --disable-silent-rules
>>> --disable-dependency-tracking
>>> --with-libtool-sysroot=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips
>>>
>>> --enable-kernel=2.6.16 --without-cvs --disable-profile --disable-debug
>>> --without-gd --enable-clocale=gnu
>>> --enable-add-ons=ports,nptl,libidn,ports
>>> --with-headers=/msp-lpggp1/mhatle/git/oe-core/build-mips32/tmp-eglibc/sysroots/qemumips/usr/include
>>>
>>> --without-selinux
>>>
>>> The system is correctly setting the target to "mips-oe-linux".
>>>
>>> I checked and bash is the same way.
>>>
>>> So the canonical arch is correct, the mips32 is only the packaging
>>> arch.  It was always intended that the packaging arch be used in full on
>>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as
>>> necessary if we expand the mips tunings.)
>>
>> I don't think such a change should be done only few days before a
>> release. Until this patch was applied, the packaging arch has always
>> been mipsel, not mips32el. Please, revert or fix this!
> 
> This is easy to change to the previous behavior...  however it was a bug
> in the original implementation.
> 
> But again, I stress nothing changed except for the packaging arch... the
> way the packages are configured, compiled, installed remain the same,
> only the package arch has changed.  The only place that may be affected
> by this is the package feed mechanism.

I think breaking package feeds in such a way is one of the worst things
to do in OE.

> To revert to the previous behavior, in the
> meta/conf/machine/include/tune-mips.inc:
> 
> --- a/meta/conf/machine/include/tune-mips32.inc
> +++ b/meta/conf/machine/include/tune-mips32.inc
> @@ -9,17 +9,17 @@ TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES",
> "mips32"
>  AVAILTUNES += "mips32 mips32el mips32-nf mips32el-nf"
> 
>  TUNE_FEATURES_tune-mips32 = "${TUNE_FEATURES_tune-mips} mips32"
> -MIPSPKGSFX_VARIANT_tune-mips32 = "mips32"
> +MIPSPKGSFX_VARIANT_tune-mips32 = "mips"
>  PACKAGE_EXTRA_ARCHS_tune-mips32 = "mips mips32"
> 
>  TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32"
> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el"
> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel"
>  PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el"
                                               ^^^^^^^^
I don't think this is correct, in all four cases, because no packages
will have that arch.

>  TUNE_FEATURES_tune-mips32-nf = "${TUNE_FEATURES_tune-mips-nf} mips32"
> -MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips32"
> +MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips"
>  PACKAGE_EXTRA_ARCHS_tune-mips32-nf = "mips-nf mips32-nf"
> 
>  TUNE_FEATURES_tune-mips32el-nf = "${TUNE_FEATURES_tune-mipsel-nf} mips32"
> -MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mips32el"
> +MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mipsel"
>  PACKAGE_EXTRA_ARCHS_tune-mips32el-nf = "mipsel-nf mips32el-nf"
> 
> Before I submit this patch though, I would like others to weigh in on
> the issue.  This was a mistake in the earlier version of the code.  The
> "variant" wasn't be set as it should have been.
> 
> The problem is that if you set the tune to "mips", you get the default
> compiler behavior.

According to the gcc docs, there is no "mips" ISA name. Valid names are:
`mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and
`mips64r2'. Therefore I don't understand why "mips" should default to
anything else, if it was an alias for mips32 before.

>  However, if you set the tune to mips32, you get
> "-march=mips32" added to your CCARGS.  This will produce slightly
> different binaries, thus "mips" and mips32" are not equal.

Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or
mips32el, so this probably broke, too.
meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still
work, e.g. EXTRA_OECONF_mipsel etc.? How about
meta/recipes-qt/qt4/qt4_arch.inc?

Whatever the answers are, the most important point is that it's the
worst point in time to do such a change. We should rather discuss it
after the release, if at all.

Regards,
Andreas
Mark Hatle - April 9, 2012, 8:25 p.m.
On 4/9/12 3:06 PM, Andreas Oberritter wrote:
> On 09.04.2012 17:17, Mark Hatle wrote:
>> On 4/8/12 4:34 PM, Andreas Oberritter wrote:
>>> On 07.04.2012 02:10, Mark Hatle wrote:
>>>> Just ran a local build with the qemumips machine, this is a standard
>>>> mips32 target.
>>>>

...

>>>> So the canonical arch is correct, the mips32 is only the packaging
>>>> arch.  It was always intended that the packaging arch be used in full on
>>>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as
>>>> necessary if we expand the mips tunings.)
>>>
>>> I don't think such a change should be done only few days before a
>>> release. Until this patch was applied, the packaging arch has always
>>> been mipsel, not mips32el. Please, revert or fix this!
>>
>> This is easy to change to the previous behavior...  however it was a bug
>> in the original implementation.
>>
>> But again, I stress nothing changed except for the packaging arch... the
>> way the packages are configured, compiled, installed remain the same,
>> only the package arch has changed.  The only place that may be affected
>> by this is the package feed mechanism.
>
> I think breaking package feeds in such a way is one of the worst things
> to do in OE.
>
>> To revert to the previous behavior, in the
>> meta/conf/machine/include/tune-mips.inc:
>>

...

>>   TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32"
>> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el"
>> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel"
>>   PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el"
>                                                 ^^^^^^^^
> I don't think this is correct, in all four cases, because no packages
> will have that arch.
>

...

>> Before I submit this patch though, I would like others to weigh in on
>> the issue.  This was a mistake in the earlier version of the code.  The
>> "variant" wasn't be set as it should have been.
>>
>> The problem is that if you set the tune to "mips", you get the default
>> compiler behavior.
>
> According to the gcc docs, there is no "mips" ISA name. Valid names are:
> `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and
> `mips64r2'. Therefore I don't understand why "mips" should default to
> anything else, if it was an alias for mips32 before.
>

We have two sets of available tunings:

"mips" and "mips32" tunings.. (add el and -nf variants)

These are -different- tunings and today the only way to notice the difference is 
based on the package arch.  The package arch is NOT the target ISA.  It's an 
arbitrary string "we" have come up with to let people know the architecture, ABI 
and optimizations used in producing specific software.  "mips" indicates that 
it's using the default mips compiler options, whatever those may be.  While 
mips32 says it is specifically tuned to the mips32 architecture settings.

I honestly have no idea what the default compiler settings for mips are, but the 
point is the tunings are different.  If you want the "MIPS" tune, you may not be 
able to run the items compiled with the -march=mips32 option.  We have to have a 
way to reconcile this.

>>   However, if you set the tune to mips32, you get
>> "-march=mips32" added to your CCARGS.  This will produce slightly
>> different binaries, thus "mips" and mips32" are not equal.
>
> Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or
> mips32el, so this probably broke, too.
> meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still
> work, e.g. EXTRA_OECONF_mipsel etc.? How about
> meta/recipes-qt/qt4/qt4_arch.inc?

Overrides are on the GNU canonical arch (TUNE_ARCH) correct?  If that is the 
case then "mips" or "mipsel" is the canonical arch.  Again, we do NOT use the 
package arch for these settings!

Below are the overrides and related elements from the bitbake.conf file:

OVERRIDES = 
"${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:forcevariable"
DISTROOVERRIDES ?= "${@d.getVar('DISTRO', True) or ''}"
MACHINEOVERRIDES ?= "${MACHINE}"

# Used by canadian-cross to handle string conversions on TARGET_ARCH where needed
TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', True).replace("_", "-")}"

TARGET_ARCH = "${TUNE_ARCH}"

So my reading of this is that, unless overriden somewhere outside of 
bitbake.conf, the override does include the TUNE_ARCH, via the TARGET_ARCH, via 
the TRANSLATED_TARGET_ARCH.

>
> Whatever the answers are, the most important point is that it's the
> worst point in time to do such a change. We should rather discuss it
> after the release, if at all.

In order to resolve this consider the following:

We have two tunes, "mips" and "mips32", the difference being the -march=mips32 
in the later case.

In order to support both tunes, we need to have a way to differentiate between 
them on a package arch basis or we end up in a situation where we have two 
packages with different contents and no way to tell them apart.

In order to reconcile the above, the three primary options are see are:

*) Define only mips or mips32 tune, but not both -- producing "mips" as the 
package arch.  (But then what do we do in the future about mips1, mips2, mips3, 
mips4, mips32r2?)

*) Revert the behavior and have two tunes that produce the identical filename 
package with different contents and deal with this in the future.

*) Keep it as it is now and produce mips and mips32 packages based on the 
specific tunings defined by the user

We have a bug, I believe we need to fix it.. first or third options "fix" the 
bug.. the second option retains the bug to be fixed in the future.

If you have an alternative to the above, I'm interested -- I just really don't 
like the "leave the bug" option.

And just to be extra clear, I consider it a defect if we can produce a package 
with the same name for two different tune settings.. (the exception being the 
hell that is ARM and thumb namings.)

--Mark

> Regards,
> Andreas
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Andreas Oberritter - April 9, 2012, 8:51 p.m.
On 09.04.2012 22:25, Mark Hatle wrote:
> On 4/9/12 3:06 PM, Andreas Oberritter wrote:
>> On 09.04.2012 17:17, Mark Hatle wrote:
>>> On 4/8/12 4:34 PM, Andreas Oberritter wrote:
>>>> On 07.04.2012 02:10, Mark Hatle wrote:
>>>>> Just ran a local build with the qemumips machine, this is a standard
>>>>> mips32 target.
>>>>>
> 
> ...
> 
>>>>> So the canonical arch is correct, the mips32 is only the packaging
>>>>> arch.  It was always intended that the packaging arch be used in
>>>>> full on
>>>>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv,
>>>>> etc as
>>>>> necessary if we expand the mips tunings.)
>>>>
>>>> I don't think such a change should be done only few days before a
>>>> release. Until this patch was applied, the packaging arch has always
>>>> been mipsel, not mips32el. Please, revert or fix this!
>>>
>>> This is easy to change to the previous behavior...  however it was a bug
>>> in the original implementation.
>>>
>>> But again, I stress nothing changed except for the packaging arch... the
>>> way the packages are configured, compiled, installed remain the same,
>>> only the package arch has changed.  The only place that may be affected
>>> by this is the package feed mechanism.
>>
>> I think breaking package feeds in such a way is one of the worst things
>> to do in OE.
>>
>>> To revert to the previous behavior, in the
>>> meta/conf/machine/include/tune-mips.inc:
>>>
> 
> ...
> 
>>>   TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32"
>>> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el"
>>> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel"
>>>   PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el"
>>                                                 ^^^^^^^^
>> I don't think this is correct, in all four cases, because no packages
>> will have that arch.
>>
> 
> ...
> 
>>> Before I submit this patch though, I would like others to weigh in on
>>> the issue.  This was a mistake in the earlier version of the code.  The
>>> "variant" wasn't be set as it should have been.
>>>
>>> The problem is that if you set the tune to "mips", you get the default
>>> compiler behavior.
>>
>> According to the gcc docs, there is no "mips" ISA name. Valid names are:
>> `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and
>> `mips64r2'. Therefore I don't understand why "mips" should default to
>> anything else, if it was an alias for mips32 before.
>>
> 
> We have two sets of available tunings:
> 
> "mips" and "mips32" tunings.. (add el and -nf variants)
> 
> These are -different- tunings and today the only way to notice the
> difference is based on the package arch.  The package arch is NOT the
> target ISA.  It's an arbitrary string "we" have come up with to let
> people know the architecture, ABI and optimizations used in producing
> specific software.  "mips" indicates that it's using the default mips
> compiler options, whatever those may be.  While mips32 says it is
> specifically tuned to the mips32 architecture settings.
> 
> I honestly have no idea what the default compiler settings for mips are,
> but the point is the tunings are different.  If you want the "MIPS"
> tune, you may not be able to run the items compiled with the
> -march=mips32 option.  We have to have a way to reconcile this.
> 
>>>   However, if you set the tune to mips32, you get
>>> "-march=mips32" added to your CCARGS.  This will produce slightly
>>> different binaries, thus "mips" and mips32" are not equal.
>>
>> Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or
>> mips32el, so this probably broke, too.
>> meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still
>> work, e.g. EXTRA_OECONF_mipsel etc.? How about
>> meta/recipes-qt/qt4/qt4_arch.inc?
> 
> Overrides are on the GNU canonical arch (TUNE_ARCH) correct?  If that is
> the case then "mips" or "mipsel" is the canonical arch.  Again, we do
> NOT use the package arch for these settings!
> 
> Below are the overrides and related elements from the bitbake.conf file:
> 
> OVERRIDES =
> "${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:forcevariable"
> 
> DISTROOVERRIDES ?= "${@d.getVar('DISTRO', True) or ''}"
> MACHINEOVERRIDES ?= "${MACHINE}"
> 
> # Used by canadian-cross to handle string conversions on TARGET_ARCH
> where needed
> TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH',
> True).replace("_", "-")}"
> 
> TARGET_ARCH = "${TUNE_ARCH}"
> 
> So my reading of this is that, unless overriden somewhere outside of
> bitbake.conf, the override does include the TUNE_ARCH, via the
> TARGET_ARCH, via the TRANSLATED_TARGET_ARCH.
> 
>>
>> Whatever the answers are, the most important point is that it's the
>> worst point in time to do such a change. We should rather discuss it
>> after the release, if at all.
> 
> In order to resolve this consider the following:
> 
> We have two tunes, "mips" and "mips32", the difference being the
> -march=mips32 in the later case.
> 
> In order to support both tunes, we need to have a way to differentiate
> between them on a package arch basis or we end up in a situation where
> we have two packages with different contents and no way to tell them apart.
> 
> In order to reconcile the above, the three primary options are see are:
> 
> *) Define only mips or mips32 tune, but not both -- producing "mips" as
> the package arch.  (But then what do we do in the future about mips1,
> mips2, mips3, mips4, mips32r2?)
> 
> *) Revert the behavior and have two tunes that produce the identical
> filename package with different contents and deal with this in the future.
> 
> *) Keep it as it is now and produce mips and mips32 packages based on
> the specific tunings defined by the user
> 
> We have a bug, I believe we need to fix it.. first or third options
> "fix" the bug.. the second option retains the bug to be fixed in the
> future.
> 
> If you have an alternative to the above, I'm interested -- I just really
> don't like the "leave the bug" option.
> 
> And just to be extra clear, I consider it a defect if we can produce a
> package with the same name for two different tune settings.. (the
> exception being the hell that is ARM and thumb namings.)

qemumips uses "mips32", so let's drop the "mips" tune for now and defer
any other changes until after the release.
Mark Hatle - April 9, 2012, 9 p.m.
On 4/9/12 3:25 PM, Mark Hatle wrote:
> On 4/9/12 3:06 PM, Andreas Oberritter wrote:
>> On 09.04.2012 17:17, Mark Hatle wrote:
>>> On 4/8/12 4:34 PM, Andreas Oberritter wrote:
>>>> On 07.04.2012 02:10, Mark Hatle wrote:
>>>>> Just ran a local build with the qemumips machine, this is a standard
>>>>> mips32 target.
>>>>>
>
> ...
>
>>>>> So the canonical arch is correct, the mips32 is only the packaging
>>>>> arch.  It was always intended that the packaging arch be used in full on
>>>>> MIPS.  (This will allow us to specify mips32r2, mipsiii, mipsiv, etc as
>>>>> necessary if we expand the mips tunings.)
>>>>
>>>> I don't think such a change should be done only few days before a
>>>> release. Until this patch was applied, the packaging arch has always
>>>> been mipsel, not mips32el. Please, revert or fix this!
>>>
>>> This is easy to change to the previous behavior...  however it was a bug
>>> in the original implementation.
>>>
>>> But again, I stress nothing changed except for the packaging arch... the
>>> way the packages are configured, compiled, installed remain the same,
>>> only the package arch has changed.  The only place that may be affected
>>> by this is the package feed mechanism.
>>
>> I think breaking package feeds in such a way is one of the worst things
>> to do in OE.
>>
>>> To revert to the previous behavior, in the
>>> meta/conf/machine/include/tune-mips.inc:
>>>
>
> ...
>
>>>    TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32"
>>> -MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el"
>>> +MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel"
>>>    PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el"
>>                                                  ^^^^^^^^
>> I don't think this is correct, in all four cases, because no packages
>> will have that arch.
>>
>
> ...
>
>>> Before I submit this patch though, I would like others to weigh in on
>>> the issue.  This was a mistake in the earlier version of the code.  The
>>> "variant" wasn't be set as it should have been.
>>>
>>> The problem is that if you set the tune to "mips", you get the default
>>> compiler behavior.
>>
>> According to the gcc docs, there is no "mips" ISA name. Valid names are:
>> `mips1', `mips2', `mips3', `mips4', `mips32', `mips32r2', `mips64' and
>> `mips64r2'. Therefore I don't understand why "mips" should default to
>> anything else, if it was an alias for mips32 before.
>>
>
> We have two sets of available tunings:
>
> "mips" and "mips32" tunings.. (add el and -nf variants)
>
> These are -different- tunings and today the only way to notice the difference is
> based on the package arch.  The package arch is NOT the target ISA.  It's an
> arbitrary string "we" have come up with to let people know the architecture, ABI
> and optimizations used in producing specific software.  "mips" indicates that
> it's using the default mips compiler options, whatever those may be.  While
> mips32 says it is specifically tuned to the mips32 architecture settings.
>
> I honestly have no idea what the default compiler settings for mips are, but the
> point is the tunings are different.  If you want the "MIPS" tune, you may not be
> able to run the items compiled with the -march=mips32 option.  We have to have a
> way to reconcile this.

I did some further digging into the GCC configuration.

The default configuration is defined in the various defines:

#define _R3000 1
#define __mips_fpr 32
#define _MIPS_ARCH_MIPS1 1
#define _MIPS_ARCH "mips1"
#define _MIPS_TUNE_MIPS1 1
#define _MIPS_TUNE "mips1"
#define __mips 1
#define _MIPS_ISA _MIPS_ISA_MIPS1

The default is defined for the MIPS1 architecture.

The -march=mips32 changes the above to:

#define __mips__ 1
#define _mips 1
#define mips 1
#define __R3000 1
#define __R3000__ 1
#define R3000 1
#define _R3000 1
#define __mips_fpr 32
#define _MIPS_ARCH_MIPS32 1
#define _MIPS_ARCH "mips32"
#define _MIPS_TUNE_MIPS32 1
#define _MIPS_TUNE "mips32"
#define __mips 32
#define __mips_isa_rev 1
#define _MIPS_ISA _MIPS_ISA_MIPS32

Internally in gcc the different between the two is processor optimizations 
change from the R3000 to the MIPS 4Kc, and PTF_AVOID_BRANCHLIKELY is defined.

    PTF_AVOID_BRANCHLIKELY
         Set if it is usually not profitable to use branch-likely instructions
         for this target, typically because the branches are always predicted
         taken and so incur a large overhead when not taken.

So there will in fact be a difference in the generated binaries.

>>>    However, if you set the tune to mips32, you get
>>> "-march=mips32" added to your CCARGS.  This will produce slightly
>>> different binaries, thus "mips" and mips32" are not equal.
>>
>> Btw, meta/recipes-core/eglibc/eglibc-ld.inc doesn't know about mips32 or
>> mips32el, so this probably broke, too.
>> meta/recipes-devtools/gdb/gdb-common.inc likewise. Do overrides still
>> work, e.g. EXTRA_OECONF_mipsel etc.? How about
>> meta/recipes-qt/qt4/qt4_arch.inc?
>
> Overrides are on the GNU canonical arch (TUNE_ARCH) correct?  If that is the
> case then "mips" or "mipsel" is the canonical arch.  Again, we do NOT use the
> package arch for these settings!
>
> Below are the overrides and related elements from the bitbake.conf file:
>
> OVERRIDES =
> "${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:forcevariable"
> DISTROOVERRIDES ?= "${@d.getVar('DISTRO', True) or ''}"
> MACHINEOVERRIDES ?= "${MACHINE}"
>
> # Used by canadian-cross to handle string conversions on TARGET_ARCH where needed
> TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', True).replace("_", "-")}"
>
> TARGET_ARCH = "${TUNE_ARCH}"
>
> So my reading of this is that, unless overriden somewhere outside of
> bitbake.conf, the override does include the TUNE_ARCH, via the TARGET_ARCH, via
> the TRANSLATED_TARGET_ARCH.
>
>>
>> Whatever the answers are, the most important point is that it's the
>> worst point in time to do such a change. We should rather discuss it
>> after the release, if at all.
>
> In order to resolve this consider the following:
>
> We have two tunes, "mips" and "mips32", the difference being the -march=mips32
> in the later case.
>
> In order to support both tunes, we need to have a way to differentiate between
> them on a package arch basis or we end up in a situation where we have two
> packages with different contents and no way to tell them apart.
>
> In order to reconcile the above, the three primary options are see are:
>
> *) Define only mips or mips32 tune, but not both -- producing "mips" as the
> package arch.  (But then what do we do in the future about mips1, mips2, mips3,
> mips4, mips32r2?)
>
> *) Revert the behavior and have two tunes that produce the identical filename
> package with different contents and deal with this in the future.
>
> *) Keep it as it is now and produce mips and mips32 packages based on the
> specific tunings defined by the user
>
> We have a bug, I believe we need to fix it.. first or third options "fix" the
> bug.. the second option retains the bug to be fixed in the future.
>
> If you have an alternative to the above, I'm interested -- I just really don't
> like the "leave the bug" option.
>
> And just to be extra clear, I consider it a defect if we can produce a package
> with the same name for two different tune settings.. (the exception being the
> hell that is ARM and thumb namings.)
>
> --Mark
>
>> Regards,
>> Andreas
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Phil Blundell - April 9, 2012, 9:03 p.m.
On Mon, 2012-04-09 at 15:25 -0500, Mark Hatle wrote:
> And just to be extra clear, I consider it a defect if we can produce a package 
> with the same name for two different tune settings.. (the exception being the 
> hell that is ARM and thumb namings.)

While you might consider that a defect (and it probably is a defensible
position to do so), it hasn't historically been considered such in OE.
The PACKAGE_ARCH value has, traditionally, been concerned purely with
ISA and ABI (i.e. answering the question "can I execute this code?")
rather than optimisations.

For example, the tune-arm926ejs.inc and tune-xscale.inc files in current
oe-core both end up setting PACKAGE_ARCH to "armv5tte" (sic).  But those
are quite different processors and have different tuning requirements,
so the binaries you get are unlikely to be the same.  If you were to
take the view that the PACKAGE_ARCH must uniquely identify one set of
binaries then obviously each of these tunings (and probably all the ARM
cpu-specific tunings) would need to set PACKAGE_ARCH to some unique
value.

p.
Mark Hatle - April 9, 2012, 9:21 p.m.
On 4/9/12 4:03 PM, Phil Blundell wrote:
> On Mon, 2012-04-09 at 15:25 -0500, Mark Hatle wrote:
>> And just to be extra clear, I consider it a defect if we can produce a package
>> with the same name for two different tune settings.. (the exception being the
>> hell that is ARM and thumb namings.)
>
> While you might consider that a defect (and it probably is a defensible
> position to do so), it hasn't historically been considered such in OE.
> The PACKAGE_ARCH value has, traditionally, been concerned purely with
> ISA and ABI (i.e. answering the question "can I execute this code?")
> rather than optimisations.
>
> For example, the tune-arm926ejs.inc and tune-xscale.inc files in current
> oe-core both end up setting PACKAGE_ARCH to "armv5tte" (sic).  But those
> are quite different processors and have different tuning requirements,
> so the binaries you get are unlikely to be the same.  If you were to
> take the view that the PACKAGE_ARCH must uniquely identify one set of
> binaries then obviously each of these tunings (and probably all the ARM
> cpu-specific tunings) would need to set PACKAGE_ARCH to some unique
> value.

I do, and thus the hell that is ARM.  I could not currently generate a single 
package feed that work would on a variety of devices (like a traditional 
workstaton/server Linux OS would.)  While this isn't a big issue in the embedded 
space where we should hopefully be aware of the tunings and configuration were 
are using, it is still a problem.  As the systems get larger, the requirement 
for common pages feeds increases, leading to the problem being, well a problem. 
  (ARM is starting to be considered for Carrier Grade systems, many of which 
have very common requirements to traditional server Linux.  A set of established 
binaries and the vendors just want to drop in optimized applications.)

On ARM what we currently have defined is:
(tune) - (package arch)

arm1136jfs - armv6-vfp
arm920t - armv4t
arm926ejs - armv5te
arm9tdmi - armv4t
armv4b - armv4b
armv4tb - armv4tb
armv4 - armv4
armv4t - armv4t
armv5b - armv5b
armv5b-vfp - armv5b-vfp
armv5eb - armv5eb
armv5eb-vfp - armv5eb-vfp
armv5ehfb-vfp - armv5ehfb-vfp
armv5ehf-vfp - armv5ehf-vfp
armv5e-vfp - armv5e-vfp
armv5hfb-vfp - armv5hfb-vfp
armv5hf-vfp - armv5hf-vfp
armv5tb - armv5tb
armv5tb-vfp - armv5b-vfp
armv5teb - armv5teb
armv5teb-vfp - armv5teb-vfp
armv5tehfb-vfp - armv5tehfb-vfp
armv5tehf-vfp - armv5tehf-vfp
armv5te - armv5te
armv5te-vfp - armv5te-vfp
armv5thfb-vfp - armv5hfb-vfp
armv5thf-vfp - armv5hf-vfp
armv5 - armv5
armv5t - armv5t
armv5t-vfp - armv5-vfp
armv5-vfp - armv5-vfp
armv6b - armv6b-vfp
armv6hfb - armv6hfb-vfp
armv6hf - armv6hf-vfp
armv6tb - armv6tb-vfp
armv6thfb - armv6thfb-vfp
armv6thf - armv6thf-vfp
armv6 - armv6-vfp
armv6t - armv6t-vfp
armv7ab-neon - armv7ab-vfp-neon
armv7ab - armv7ab-vfp
armv7ahfb-neon - armv7ahfb-vfp-neon
armv7ahfb - armv7ahfb-vfp
armv7ahf-neon - armv7ahf-vfp-neon
armv7ahf - armv7ahf-vfp
armv7a-neon - armv7a-vfp-neon
armv7atb-neon - armv7ab-vfp-neon
armv7atb - armv7ab-vfp
armv7athfb - armv7ahfb-vfp
armv7athf-neon - armv7ahf-vfp-neon
armv7athf - armv7ahf-vfp
armv7a - armv7a-vfp
armv7at-neon - armv7a-vfp-neon
armv7at - armv7a-vfp
cortexa8hf-neon - armv7ahf-vfp-neon
cortexa8hf - armv7ahf-vfp
cortexa8-neon - armv7a-vfp-neon
cortexa8thf - armv7ahf-vfp
cortexa8 - armv7a-vfp
cortexa8t - armv7a-vfp
cortexa9hf-neon - armv7ahf-vfp-neon
cortexa9hf - armv7ahf-vfp
cortexa9-neon - armv7a-vfp-neon
cortexa9thf - armv7ahf-vfp
cortexa9 - armv7a-vfp
cortexa9t - armv7a-vfp
cortexm1 - armv7a-vfp
cortexm3 - armv7m-vfp
cortexr4 - armv7r-vfp
ep9312 - ep9312
iwmmxt - iwmmxt
strongarm - armv4

Add in to that one of the tunings -- not indicated by the package arch of thumb 
enabled or not, and its difficult to know exactly what is in a package without 
extracting and examining it.  But I consider this to be a quirk of the ARM 
architecture as implemented in OE.

--Mark

> p.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Phil Blundell - April 9, 2012, 9:30 p.m.
On Mon, 2012-04-09 at 16:21 -0500, Mark Hatle wrote:
> I do, and thus the hell that is ARM.  I could not currently generate a single 
> package feed that work would on a variety of devices (like a traditional 
> workstaton/server Linux OS would.) 

Well, actually, you could in fact do exactly that.  What you couldn't
necessarily do with the tunings as they exist right now is generate a
package feed which is optimised for (as opposed to "works on") all those
devices.  But it isn't clear to me that you could do that with a
"traditional workstaton/server" kind of distribution either.  In the x86
world, for example, the majority of the big distros do not bother to
ship individually-tuned binaries for different processor types,
certainly not for the entire distribution.

>Add in to that one of the tunings -- not indicated by the package arch
>of thumb enabled or not

There are multiple reasons why this isn't indicated by the PACKAGE_ARCH.
Firstly, it's irrelevant: on v5T or newer, the question of whether a
given package is using Thumb-state or not has no ABI impact and there is
no reason for anyone to care at a compatibility level.  Second, it may
be unpredictable: the compiler is at liberty (although current versions
of gcc don't exploit this latitude) to switch arbitrarily between
ARM-state and Thumb-state as it sees fit to get the best performance.
And thirdly, it's just another piece of distro policy in the same way as
compiling for -O2 vs -Os (which we also don't encode into PACKAGE_ARCH)
is.

p.
Mark Hatle - April 9, 2012, 9:44 p.m.
On 4/9/12 4:30 PM, Phil Blundell wrote:
> On Mon, 2012-04-09 at 16:21 -0500, Mark Hatle wrote:
>> I do, and thus the hell that is ARM.  I could not currently generate a single
>> package feed that work would on a variety of devices (like a traditional
>> workstaton/server Linux OS would.)
>
> Well, actually, you could in fact do exactly that.  What you couldn't
> necessarily do with the tunings as they exist right now is generate a
> package feed which is optimised for (as opposed to "works on") all those
> devices.  But it isn't clear to me that you could do that with a
> "traditional workstaton/server" kind of distribution either.  In the x86
> world, for example, the majority of the big distros do not bother to
> ship individually-tuned binaries for different processor types,
> certainly not for the entire distribution.

Depends on the distribution and reasons for these feeds.  What is typical is 
that a base distribution will be generated for a common compatible (reasonable) 
architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, 
etc) for the target arch, i.e. armv7a.  Then you have a couple of packages 
hand-tuned for size, speed, or other that define or not thumb and add even a 
higher level of optimization.  It's possible, folks do it today.. but it's not 
always obvious.  (I have existing customers today that run a mix like I 
described through their own package feed like system.  They really don't care at 
all that the core system is tuned for a given processor -- they only care that 
their specific applications and certain areas are specifically tuned to their 
use-cases.)  Note, this is not what I would consider a typical use-case!

>> Add in to that one of the tunings -- not indicated by the package arch
>> of thumb enabled or not
>
> There are multiple reasons why this isn't indicated by the PACKAGE_ARCH.
> Firstly, it's irrelevant: on v5T or newer, the question of whether a
> given package is using Thumb-state or not has no ABI impact and there is
> no reason for anyone to care at a compatibility level.  Second, it may
> be unpredictable: the compiler is at liberty (although current versions
> of gcc don't exploit this latitude) to switch arbitrarily between
> ARM-state and Thumb-state as it sees fit to get the best performance.
> And thirdly, it's just another piece of distro policy in the same way as
> compiling for -O2 vs -Os (which we also don't encode into PACKAGE_ARCH)
> is.

I agree, on ARM the tunings and optimizations between regular and thumb do not 
impact the ABI what-so-ever.  And so far compilers have to be explicitly set to 
do thumb or tranditional ARM mode.. so in the end developers are looking into 
the performance and size impacts of each of these configuration and making 
changes as they see fit to best meet their needs.  These are unique cases 
though, the majority of the software built for the core OS uses a single policy 
-- it's when something needs to be further optimized that this comes into play.

At this point, I'd "like" to better differentiate the ARM package arches..  I 
don't care so much about the thumb enabled or not.. but the other tune settings 
are things I do care about.  I started to change that for the last update and 
decided it was a rat-hole I was not willing to go down at this point.  At some 
point in the future, I will look at, and document the differences in the tunings 
according to GCC configurations -- to get a good idea of what is and isn't 
producing the same binaries based on various arch and tune settings.

--Mark

> p.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Khem Raj - April 9, 2012, 10:19 p.m.
On Mon, Apr 9, 2012 at 1:25 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>
> In order to reconcile the above, the three primary options are see are:
>
> *) Define only mips or mips32 tune, but not both -- producing "mips" as the
> package arch.  (But then what do we do in the future about mips1, mips2,
> mips3, mips4, mips32r2?)
>
> *) Revert the behavior and have two tunes that produce the identical
> filename package with different contents and deal with this in the future.
>
> *) Keep it as it is now and produce mips and mips32 packages based on the
> specific tunings defined by the user

I think situation is not something I like but

I would think that 1st option is better. Any machine that has been using
mips32 tuning accidently could now use mips tune files and get same PKG_ARCH
and we move mips32 tune into mips tune and hereon say that mips 32bit in OE
defaults to -march=mips32

we could also change compiler defaults to use -march=mips32 to match
the baseconfig and since qemumips has been using mips32 anyway that sort
of is base mips arch.

>
> We have a bug, I believe we need to fix it.. first or third options "fix"
> the bug.. the second option retains the bug to be fixed in the future.
>
> If you have an alternative to the above, I'm interested -- I just really
> don't like the "leave the bug" option.
>
> And just to be extra clear, I consider it a defect if we can produce a
> package with the same name for two different tune settings.. (the exception
> being the hell that is ARM and thumb namings.)
Phil Blundell - April 10, 2012, 9:23 a.m.
On Mon, 2012-04-09 at 16:44 -0500, Mark Hatle wrote:
> Depends on the distribution and reasons for these feeds.  What is typical is 
> that a base distribution will be generated for a common compatible (reasonable) 
> architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl, 
> etc) for the target arch, i.e. armv7a.  Then you have a couple of packages 
> hand-tuned for size, speed, or other that define or not thumb and add even a 
> higher level of optimization.  It's possible, folks do it today.. but it's not 
> always obvious.  (I have existing customers today that run a mix like I 
> described through their own package feed like system.  They really don't care at 
> all that the core system is tuned for a given processor -- they only care that 
> their specific applications and certain areas are specifically tuned to their 
> use-cases.)  Note, this is not what I would consider a typical use-case!

Sorry, I'm not quite sure I understand what point you're trying to make
here.  Are you describing what your customers are currently doing with
OE, or are you saying that this is something that they would like to do
with OE but don't feel they are able to at present, or something else?

I'm still not entirely clear on what you feel is broken about the
current state of the ARM tunings.  What exactly is the scenario that
works with the "traditional workstaton/server Linux OS" and can't be
replicated with OE?

But, all that aside, it seems ultimately that the exact way the
PACKAGE_ARCHs are structured ought to be a DISTRO policy decision and
not something that's mandated by the underlying infrastructure.  That
would perhaps remove some of the need for tinkering with these things in
oe-core itself.

p.
Mark Hatle - April 10, 2012, 5:39 p.m.
On 4/10/12 4:23 AM, Phil Blundell wrote:
> On Mon, 2012-04-09 at 16:44 -0500, Mark Hatle wrote:
>> Depends on the distribution and reasons for these feeds.  What is typical is
>> that a base distribution will be generated for a common compatible (reasonable)
>> architecture.. i.e. armv5 -- with specific optimized package (glibc, openssl,
>> etc) for the target arch, i.e. armv7a.  Then you have a couple of packages
>> hand-tuned for size, speed, or other that define or not thumb and add even a
>> higher level of optimization.  It's possible, folks do it today.. but it's not
>> always obvious.  (I have existing customers today that run a mix like I
>> described through their own package feed like system.  They really don't care at
>> all that the core system is tuned for a given processor -- they only care that
>> their specific applications and certain areas are specifically tuned to their
>> use-cases.)  Note, this is not what I would consider a typical use-case!

> Sorry, I'm not quite sure I understand what point you're trying to make
> here.  Are you describing what your customers are currently doing with
> OE, or are you saying that this is something that they would like to do
> with OE but don't feel they are able to at present, or something else?


The company I work for has an existing product that does not use OE.  The 
customers using this have requested from us packages tagged with different 
package architectures to indicate the tuning and optimization information so 
that they can create multiple boards with the same software running on them. 
(This is closer to the traditional workstation/server model in my experience.) 
The multiple boards have a common set of OS applications that run on them. 
ARMv5 for the most part.  The customers then have optimized applications with or 
without thumb, and optimized for a variety of different ARM parts.  They then 
use the binary packages to assemble the filesystems, and perform field upgrades, 
on these boards as they are put into use.  The installation system uses a best 
to least best match when doing assembly actions.  So if the part is an ARMv7a, 
it will first look for ARMv7a w/ thumb, vfp and neon, not finding that, ARMv7a 
w/ thumb and vfp, then ARMv7a w/ thumb, then ARMv5 w/ thumb and vfp, then ARMv5 
w/ thumb, and finally fall back to the ARMv5 binaries.

> I'm still not entirely clear on what you feel is broken about the
> current state of the ARM tunings.  What exactly is the scenario that
> works with the "traditional workstaton/server Linux OS" and can't be
> replicated with OE?

With the current implementation, there is no package differentiation between 
thumb and non-thumb binaries.  I accept why this is in OE-core and I can live 
with that.  However, there are other binaries that are theoretically optimized 
from specific Cortex models to the more generic ARMv7a tunings.  Currently they 
all use the same package arch, which means I can't tell which CPU they're really 
for -- and this model (above) of best to least best match doesn't work.

On a pure embedded model, I doubt anyone would do this.  Thus it is a fairly 
unique embedded use-case, but a common Workstation/Server use case that is being 
replicated.

> But, all that aside, it seems ultimately that the exact way the
> PACKAGE_ARCHs are structured ought to be a DISTRO policy decision and
> not something that's mandated by the underlying infrastructure.  That
> would perhaps remove some of the need for tinkering with these things in
> oe-core itself.

I intend, after this release, to propose changes to differentiate the models in 
oe-core.  If the oe-core folks do not feel this is necessary, they I will 
maintain them on my own as I feel necessary to cover the above use-case.

I can very much understand that in OE, for ARM specifically the package arch is 
simply indicating basic compatibility and not ABI & ISA & Optimization like it 
is on other architectures.

--Mark

> p.
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Phil Blundell - April 10, 2012, 7:33 p.m.
On Tue, 2012-04-10 at 12:39 -0500, Mark Hatle wrote:
> The installation system uses a best 
> to least best match when doing assembly actions.  So if the part is an ARMv7a, 
> it will first look for ARMv7a w/ thumb, vfp and neon, not finding that, ARMv7a 
> w/ thumb and vfp, then ARMv7a w/ thumb, then ARMv5 w/ thumb and vfp, then ARMv5 
> w/ thumb, and finally fall back to the ARMv5 binaries.

This sounds like exactly the behaviour you would get with the current
ARM tunings (except the Thumb bit which, as previously discussed, I
think is somewhat misguided in the first place).  The existing ARM
tunings do seem to correctly encode VFP and Neon-ness.  

Which part isn't working for you?  Maybe you could give a concrete
example of where exactly it falls down.

> I can very much understand that in OE, for ARM specifically the package arch is 
> simply indicating basic compatibility and not ABI & ISA & Optimization like it 
> is on other architectures.

Well, I would consider "ABI & ISA" to be a fairly big part of "basic
compatibility".  It is true that we don't currently encode
optimisations, but as I previously mentioned I don't think many (perhaps
any) other distributions do that either, and it's perhaps debatable
whether it would be a very useful thing to do in the general case.  For
individual packages you can obviously force the issue in your DISTRO
configuration anyway.

p.

Patch

--- a/meta/conf/machine/include/tune-mips32.inc
+++ b/meta/conf/machine/include/tune-mips32.inc
@@ -9,17 +9,17 @@  TUNE_CCARGS += "${@bb.utils.contains("TUNE_FEATURES", "mips32"
  AVAILTUNES += "mips32 mips32el mips32-nf mips32el-nf"

  TUNE_FEATURES_tune-mips32 = "${TUNE_FEATURES_tune-mips} mips32"
-MIPSPKGSFX_VARIANT_tune-mips32 = "mips32"
+MIPSPKGSFX_VARIANT_tune-mips32 = "mips"
  PACKAGE_EXTRA_ARCHS_tune-mips32 = "mips mips32"

  TUNE_FEATURES_tune-mips32el = "${TUNE_FEATURES_tune-mipsel} mips32"
-MIPSPKGSFX_VARIANT_tune-mips32el = "mips32el"
+MIPSPKGSFX_VARIANT_tune-mips32el = "mipsel"
  PACKAGE_EXTRA_ARCHS_tune-mips32el = "mipsel mips32el"

  TUNE_FEATURES_tune-mips32-nf = "${TUNE_FEATURES_tune-mips-nf} mips32"
-MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips32"
+MIPSPKGSFX_VARIANT_tune-mips32-nf = "mips"
  PACKAGE_EXTRA_ARCHS_tune-mips32-nf = "mips-nf mips32-nf"

  TUNE_FEATURES_tune-mips32el-nf = "${TUNE_FEATURES_tune-mipsel-nf} mips32"
-MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mips32el"
+MIPSPKGSFX_VARIANT_tune-mips32el-nf = "mipsel"
  PACKAGE_EXTRA_ARCHS_tune-mips32el-nf = "mipsel-nf mips32el-nf"