[1/1] buildtools-tarball: add nativesdk-libxml2-utils

Submitted by Qi.Chen@windriver.com on July 8, 2019, 7:37 a.m. | Patch ID: 162870

Details

Message ID 0dee845494880f0c201e79d6ee0f50390f183465.1562570957.git.Qi.Chen@windriver.com
State New
Headers show

Commit Message

Qi.Chen@windriver.com July 8, 2019, 7:37 a.m.
For build-sysroots.bb, the xmlcatalog would not be in its
staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
   IMAGE_INSTALL_append = " btrfs-tools"
   DISTRO_FEATURES_append = " api-documentation"
   INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball
to ensure the existence of xmlcatalog. Also add it
to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/conf/bitbake.conf                       | 3 +++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 2 files changed, 4 insertions(+)

Patch hide | download patch | download mbox

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 5e93f5c..2f64eae 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -517,6 +517,9 @@  HOSTTOOLS_NONFATAL += "scp"
 # Link to git-lfs if present
 HOSTTOOLS_NONFATAL += "git-lfs"
 
+# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"
+
 CCACHE ??= ""
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
index 91df6f1..d39d6f7 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -25,6 +25,7 @@  TOOLCHAIN_HOST_TASK ?= "\
     nativesdk-texinfo \
     nativesdk-libnss-nis \
     nativesdk-rpcsvc-proto \
+    nativesdk-libxml2-utils \
     "
 
 MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"

Comments

Armin Kuster July 8, 2019, 3:55 p.m.
On 7/8/19 12:37 AM, Chen Qi wrote:
> For build-sysroots.bb, the xmlcatalog would not be in its
> staging directory. Causing the following error for eSDK.
>
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function failed: do_build_native_sysroot
>
> The problem could be reproduced by the following steps.
> 1. Add in local.conf:
>    IMAGE_INSTALL_append = " btrfs-tools"
>    DISTRO_FEATURES_append = " api-documentation"
>    INHERIT += "testsdk"
> 2. bitbake core-image-minimal -c populate_sdk_ext
> 3. bitbake core-image-minimal -c testsdkext

should these steps be added to eSDK tests?

- armin
>
> So we add nativesdk-libxml2-utils to buildtools-tarball
> to ensure the existence of xmlcatalog. Also add it
> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  meta/conf/bitbake.conf                       | 3 +++
>  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>  2 files changed, 4 insertions(+)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 5e93f5c..2f64eae 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>  # Link to git-lfs if present
>  HOSTTOOLS_NONFATAL += "git-lfs"
>  
> +# build-sysroot needs xmlcatalog in order for eSDK installation
> +HOSTTOOLS_NONFATAL += "xmlcatalog"
> +
>  CCACHE ??= ""
>  
>  TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
> index 91df6f1..d39d6f7 100644
> --- a/meta/recipes-core/meta/buildtools-tarball.bb
> +++ b/meta/recipes-core/meta/buildtools-tarball.bb
> @@ -25,6 +25,7 @@ TOOLCHAIN_HOST_TASK ?= "\
>      nativesdk-texinfo \
>      nativesdk-libnss-nis \
>      nativesdk-rpcsvc-proto \
> +    nativesdk-libxml2-utils \
>      "
>  
>  MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"
Richard Purdie July 8, 2019, 4:44 p.m.
On Mon, 2019-07-08 at 08:55 -0700, akuster808 wrote:
> 
> On 7/8/19 12:37 AM, Chen Qi wrote:
> > For build-sysroots.bb, the xmlcatalog would not be in its
> > staging directory. Causing the following error for eSDK.
> > 
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> > '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
> > docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
> > 127.
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> > failed: do_build_native_sysroot
> > 
> > The problem could be reproduced by the following steps.
> > 1. Add in local.conf:
> >    IMAGE_INSTALL_append = " btrfs-tools"
> >    DISTRO_FEATURES_append = " api-documentation"
> >    INHERIT += "testsdk"
> > 2. bitbake core-image-minimal -c populate_sdk_ext
> > 3. bitbake core-image-minimal -c testsdkext
> 
> should these steps be added to eSDK tests?

api-documentation is only tested in a single build in qa-extras and
esdk is only tested in the standard config.

There are many different variants we don't test, its hard to tell which
ones we should be adding and are worth the extra build times...

Cheers,

Richard
Richard Purdie July 8, 2019, 4:47 p.m.
On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
> For build-sysroots.bb, the xmlcatalog would not be in its
> staging directory. Causing the following error for eSDK.
> 
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
> 127.
> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> failed: do_build_native_sysroot
> 
> The problem could be reproduced by the following steps.
> 1. Add in local.conf:
>    IMAGE_INSTALL_append = " btrfs-tools"
>    DISTRO_FEATURES_append = " api-documentation"
>    INHERIT += "testsdk"
> 2. bitbake core-image-minimal -c populate_sdk_ext
> 3. bitbake core-image-minimal -c testsdkext
> 
> So we add nativesdk-libxml2-utils to buildtools-tarball
> to ensure the existence of xmlcatalog. Also add it
> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
> 
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  meta/conf/bitbake.conf                       | 3 +++
>  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>  2 files changed, 4 insertions(+)
> 
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 5e93f5c..2f64eae 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>  # Link to git-lfs if present
>  HOSTTOOLS_NONFATAL += "git-lfs"
>  
> +# build-sysroot needs xmlcatalog in order for eSDK installation
> +HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
grow to contain far too much random stuff which could impact
reproduciblity.

Is there some other way we could fix this? It still feels like a
dependency problem which we chould fix by adding a missing dependency
although I appreciate its far from being that simple.

I wonder if there is a way we could teach build-sysroots to be cleverer
about dependencies?

Cheers,

Richard
Ross Burton July 8, 2019, 6:45 p.m.
Agreed, making it a non-fatal dependency just means that some eSDKs
break, not all.  There needs to be a better fix.

Ross

On Mon, 8 Jul 2019 at 17:53, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
> > For build-sysroots.bb, the xmlcatalog would not be in its
> > staging directory. Causing the following error for eSDK.
> >
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> > '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
> > docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
> > 127.
> > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> > failed: do_build_native_sysroot
> >
> > The problem could be reproduced by the following steps.
> > 1. Add in local.conf:
> >    IMAGE_INSTALL_append = " btrfs-tools"
> >    DISTRO_FEATURES_append = " api-documentation"
> >    INHERIT += "testsdk"
> > 2. bitbake core-image-minimal -c populate_sdk_ext
> > 3. bitbake core-image-minimal -c testsdkext
> >
> > So we add nativesdk-libxml2-utils to buildtools-tarball
> > to ensure the existence of xmlcatalog. Also add it
> > to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
> >
> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > ---
> >  meta/conf/bitbake.conf                       | 3 +++
> >  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
> >  2 files changed, 4 insertions(+)
> >
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 5e93f5c..2f64eae 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
> >  # Link to git-lfs if present
> >  HOSTTOOLS_NONFATAL += "git-lfs"
> >
> > +# build-sysroot needs xmlcatalog in order for eSDK installation
> > +HOSTTOOLS_NONFATAL += "xmlcatalog"
>
> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
> grow to contain far too much random stuff which could impact
> reproduciblity.
>
> Is there some other way we could fix this? It still feels like a
> dependency problem which we chould fix by adding a missing dependency
> although I appreciate its far from being that simple.
>
> I wonder if there is a way we could teach build-sysroots to be cleverer
> about dependencies?
>
> Cheers,
>
> Richard
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Qi.Chen@windriver.com July 9, 2019, 1:38 a.m.
On 07/09/2019 12:47 AM, Richard Purdie wrote:
> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>> For build-sysroots.bb, the xmlcatalog would not be in its
>> staging directory. Causing the following error for eSDK.
>>
>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
>> 127.
>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>> failed: do_build_native_sysroot
>>
>> The problem could be reproduced by the following steps.
>> 1. Add in local.conf:
>>     IMAGE_INSTALL_append = " btrfs-tools"
>>     DISTRO_FEATURES_append = " api-documentation"
>>     INHERIT += "testsdk"
>> 2. bitbake core-image-minimal -c populate_sdk_ext
>> 3. bitbake core-image-minimal -c testsdkext
>>
>> So we add nativesdk-libxml2-utils to buildtools-tarball
>> to ensure the existence of xmlcatalog. Also add it
>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> ---
>>   meta/conf/bitbake.conf                       | 3 +++
>>   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>   2 files changed, 4 insertions(+)
>>
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index 5e93f5c..2f64eae 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>   # Link to git-lfs if present
>>   HOSTTOOLS_NONFATAL += "git-lfs"
>>   
>> +# build-sysroot needs xmlcatalog in order for eSDK installation
>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
> grow to contain far too much random stuff which could impact
> reproduciblity.
>
> Is there some other way we could fix this? It still feels like a
> dependency problem which we chould fix by adding a missing dependency
> although I appreciate its far from being that simple.
>
> I wonder if there is a way we could teach build-sysroots to be cleverer
> about dependencies?

I'll try to implement it and send out a new patch.

Regards,
Chen Qi

> Cheers,
>
> Richard
>
>
Qi.Chen@windriver.com July 9, 2019, 1:40 a.m.
On 07/09/2019 02:45 AM, Burton, Ross wrote:
> Agreed, making it a non-fatal dependency just means that some eSDKs
> break, not all.  There needs to be a better fix.

But it's already in buildtools, and eSDK installation process involves 
extracting buildtools and sourcing its script.
Under which circumstances will we break some eSDKs?
Anyway, I'm trying to do something about build-sysroots recipe and will 
send out a new patch.

Regards,
Chen Qi

> Ross
>
> On Mon, 8 Jul 2019 at 17:53, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>>> For build-sysroots.bb, the xmlcatalog would not be in its
>>> staging directory. Causing the following error for eSDK.
>>>
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
>>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
>>> 127.
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>>> failed: do_build_native_sysroot
>>>
>>> The problem could be reproduced by the following steps.
>>> 1. Add in local.conf:
>>>     IMAGE_INSTALL_append = " btrfs-tools"
>>>     DISTRO_FEATURES_append = " api-documentation"
>>>     INHERIT += "testsdk"
>>> 2. bitbake core-image-minimal -c populate_sdk_ext
>>> 3. bitbake core-image-minimal -c testsdkext
>>>
>>> So we add nativesdk-libxml2-utils to buildtools-tarball
>>> to ensure the existence of xmlcatalog. Also add it
>>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>>
>>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>> ---
>>>   meta/conf/bitbake.conf                       | 3 +++
>>>   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>>   2 files changed, 4 insertions(+)
>>>
>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>> index 5e93f5c..2f64eae 100644
>>> --- a/meta/conf/bitbake.conf
>>> +++ b/meta/conf/bitbake.conf
>>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>>   # Link to git-lfs if present
>>>   HOSTTOOLS_NONFATAL += "git-lfs"
>>>
>>> +# build-sysroot needs xmlcatalog in order for eSDK installation
>>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
>> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
>> grow to contain far too much random stuff which could impact
>> reproduciblity.
>>
>> Is there some other way we could fix this? It still feels like a
>> dependency problem which we chould fix by adding a missing dependency
>> although I appreciate its far from being that simple.
>>
>> I wonder if there is a way we could teach build-sysroots to be cleverer
>> about dependencies?
>>
>> Cheers,
>>
>> Richard
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Qi.Chen@windriver.com July 9, 2019, 9:36 a.m.
On 07/09/2019 09:38 AM, ChenQi wrote:
> On 07/09/2019 12:47 AM, Richard Purdie wrote:
>> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>>> For build-sysroots.bb, the xmlcatalog would not be in its
>>> staging directory. Causing the following error for eSDK.
>>>
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-
>>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
>>> 127.
>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>>> failed: do_build_native_sysroot
>>>
>>> The problem could be reproduced by the following steps.
>>> 1. Add in local.conf:
>>>     IMAGE_INSTALL_append = " btrfs-tools"
>>>     DISTRO_FEATURES_append = " api-documentation"
>>>     INHERIT += "testsdk"
>>> 2. bitbake core-image-minimal -c populate_sdk_ext
>>> 3. bitbake core-image-minimal -c testsdkext
>>>
>>> So we add nativesdk-libxml2-utils to buildtools-tarball
>>> to ensure the existence of xmlcatalog. Also add it
>>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>>
>>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>> ---
>>>   meta/conf/bitbake.conf                       | 3 +++
>>>   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>>   2 files changed, 4 insertions(+)
>>>
>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>> index 5e93f5c..2f64eae 100644
>>> --- a/meta/conf/bitbake.conf
>>> +++ b/meta/conf/bitbake.conf
>>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>>   # Link to git-lfs if present
>>>   HOSTTOOLS_NONFATAL += "git-lfs"
>>>   +# build-sysroot needs xmlcatalog in order for eSDK installation
>>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
>> I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
>> grow to contain far too much random stuff which could impact
>> reproduciblity.
>>
>> Is there some other way we could fix this? It still feels like a
>> dependency problem which we chould fix by adding a missing dependency
>> although I appreciate its far from being that simple.
>>
>> I wonder if there is a way we could teach build-sysroots to be cleverer
>> about dependencies?
>
> I'll try to implement it and send out a new patch.
>

Hi Richard and Ross,

Unfortunately I cannot figure out a good way to implement this.
Could you please help giving me some hints?

Best Regards,
Chen Qi

> Regards,
> Chen Qi
>
>> Cheers,
>>
>> Richard
>>
>>
>
Richard Purdie July 9, 2019, 10:18 a.m.
On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
> On 07/09/2019 09:38 AM, ChenQi wrote:
> > On 07/09/2019 12:47 AM, Richard Purdie wrote:
> > > On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
> > > > For build-sysroots.bb, the xmlcatalog would not be in its
> > > > staging directory. Causing the following error for eSDK.
> > > > 
> > > > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
> > > > '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst
> > > > -
> > > > docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit
> > > > status
> > > > 127.
> > > > ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
> > > > failed: do_build_native_sysroot
> > > > 
> > > > The problem could be reproduced by the following steps.
> > > > 1. Add in local.conf:
> > > >     IMAGE_INSTALL_append = " btrfs-tools"
> > > >     DISTRO_FEATURES_append = " api-documentation"
> > > >     INHERIT += "testsdk"
> > > > 2. bitbake core-image-minimal -c populate_sdk_ext
> > > > 3. bitbake core-image-minimal -c testsdkext
> > > > 
> > > > So we add nativesdk-libxml2-utils to buildtools-tarball
> > > > to ensure the existence of xmlcatalog. Also add it
> > > > to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
> > > > 
> > > > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> > > > ---
> > > >   meta/conf/bitbake.conf                       | 3 +++
> > > >   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
> > > >   2 files changed, 4 insertions(+)
> > > > 
> > > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > > > index 5e93f5c..2f64eae 100644
> > > > --- a/meta/conf/bitbake.conf
> > > > +++ b/meta/conf/bitbake.conf
> > > > @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
> > > >   # Link to git-lfs if present
> > > >   HOSTTOOLS_NONFATAL += "git-lfs"
> > > >   +# build-sysroot needs xmlcatalog in order for eSDK
> > > > installation
> > > > +HOSTTOOLS_NONFATAL += "xmlcatalog"
> > > I don't mind the buildtools-tarball change but HOSTTOOLS is
> > > starting to
> > > grow to contain far too much random stuff which could impact
> > > reproduciblity.
> > > 
> > > Is there some other way we could fix this? It still feels like a
> > > dependency problem which we chould fix by adding a missing
> > > dependency
> > > although I appreciate its far from being that simple.
> > > 
> > > I wonder if there is a way we could teach build-sysroots to be
> > > cleverer
> > > about dependencies?
> > 
> > I'll try to implement it and send out a new patch.
> 
> Unfortunately I cannot figure out a good way to implement this.
> Could you please help giving me some hints?

What happens if we add:

${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'nativesdk-libxml2', '', d)} \

to
`
meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb 

?

Cheers,

Richard
Qi.Chen@windriver.com July 10, 2019, 3:06 a.m.
On 07/09/2019 06:18 PM, Richard Purdie wrote:
> On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
>> On 07/09/2019 09:38 AM, ChenQi wrote:
>>> On 07/09/2019 12:47 AM, Richard Purdie wrote:
>>>> On Mon, 2019-07-08 at 15:37 +0800, Chen Qi wrote:
>>>>> For build-sysroots.bb, the xmlcatalog would not be in its
>>>>> staging directory. Causing the following error for eSDK.
>>>>>
>>>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command
>>>>> '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst
>>>>> -
>>>>> docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit
>>>>> status
>>>>> 127.
>>>>> ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
>>>>> failed: do_build_native_sysroot
>>>>>
>>>>> The problem could be reproduced by the following steps.
>>>>> 1. Add in local.conf:
>>>>>      IMAGE_INSTALL_append = " btrfs-tools"
>>>>>      DISTRO_FEATURES_append = " api-documentation"
>>>>>      INHERIT += "testsdk"
>>>>> 2. bitbake core-image-minimal -c populate_sdk_ext
>>>>> 3. bitbake core-image-minimal -c testsdkext
>>>>>
>>>>> So we add nativesdk-libxml2-utils to buildtools-tarball
>>>>> to ensure the existence of xmlcatalog. Also add it
>>>>> to HOSTTOOLS_NONFATAL so it could be seen by bitbake.
>>>>>
>>>>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>>>> ---
>>>>>    meta/conf/bitbake.conf                       | 3 +++
>>>>>    meta/recipes-core/meta/buildtools-tarball.bb | 1 +
>>>>>    2 files changed, 4 insertions(+)
>>>>>
>>>>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>>>>> index 5e93f5c..2f64eae 100644
>>>>> --- a/meta/conf/bitbake.conf
>>>>> +++ b/meta/conf/bitbake.conf
>>>>> @@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
>>>>>    # Link to git-lfs if present
>>>>>    HOSTTOOLS_NONFATAL += "git-lfs"
>>>>>    +# build-sysroot needs xmlcatalog in order for eSDK
>>>>> installation
>>>>> +HOSTTOOLS_NONFATAL += "xmlcatalog"
>>>> I don't mind the buildtools-tarball change but HOSTTOOLS is
>>>> starting to
>>>> grow to contain far too much random stuff which could impact
>>>> reproduciblity.
>>>>
>>>> Is there some other way we could fix this? It still feels like a
>>>> dependency problem which we chould fix by adding a missing
>>>> dependency
>>>> although I appreciate its far from being that simple.
>>>>
>>>> I wonder if there is a way we could teach build-sysroots to be
>>>> cleverer
>>>> about dependencies?
>>> I'll try to implement it and send out a new patch.
>> Unfortunately I cannot figure out a good way to implement this.
>> Could you please help giving me some hints?
> What happens if we add:
>
> ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'nativesdk-libxml2', '', d)} \
>
> to
> `
> meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
>
> ?

This does not work. Using nativesdk-libxml2-utils does not work too.
Both buildtools and eSDK do not use this packagegroup.

I've sent out a V2 of this patch, with change to put the 
'libxml2-native' dependency in xmlcatalog.bbclass.
In this way, we can ensure that the xmlcatalog is from our recipe. In 
normal build, it's from libxml2-native; in build-sysroots in eSDK, it's 
from nativesdk-libxml2-utils. Do you think this is acceptable?

Best Regards,
Chen Qi

> Cheers,
>
> Richard
>
>
Richard Purdie July 10, 2019, 9:08 a.m.
On Wed, 2019-07-10 at 11:06 +0800, ChenQi wrote:
> On 07/09/2019 06:18 PM, Richard Purdie wrote:
> > On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
> > What happens if we add:
> > 
> > ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation',
> > 'nativesdk-libxml2', '', d)} \
> > 
> > to
> > `
> > meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb
> > 
> > ?
> 
> This does not work. Using nativesdk-libxml2-utils does not work too.
> Both buildtools and eSDK do not use this packagegroup.

Ok, I'll need to spend more time looking at the problem then :(.

> I've sent out a V2 of this patch, with change to put the 
> 'libxml2-native' dependency in xmlcatalog.bbclass.
> In this way, we can ensure that the xmlcatalog is from our recipe.
> In normal build, it's from libxml2-native; in build-sysroots in eSDK,
> it's from nativesdk-libxml2-utils. Do you think this is acceptable?

The class change is fine and I'll queue that.

We are not adding xmlcatalog to HOSTTOOLS_NONFATAL however, that is
simply not an option or the right way to fix this.

Cheers,

Richard