Patchwork [0/5] refactor the archive*.bbcalss

login
register
mail settings
Submitter Robert Yang
Date Jan. 13, 2014, 4:20 p.m.
Message ID <cover.1389629682.git.liezhi.yang@windriver.com>
Download mbox
Permalink /patch/64597/
State New
Headers show

Pull-request

git://git.pokylinux.org/poky-contrib rbt/archiver

Comments

Martin Jansa - Jan. 13, 2014, 10:21 a.m.
On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> * The archive*.bbclass didn't work, and there were a few problems, for
>   example:
>   1) There were a few duplicated code
>   2) There was no src_dir.org (or orig), but the diff command still use
>      it, and it is not easy to fix this issue if we don't change a lot
>      of the code.
>   3) It didn't archive the source for the native or gcc
>   4) The work flow is not very well
>   5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>   6) And others ...
> 
> * So that we have to refactor it, the benefits are:
>   1) Fix the problems and make it work well.
>   2) Reduce more than 400 lines in total.
>   3) Make it easy to use.

Have you seen
http://lists.openembedded.org/pipermail/openembedded-core/2013-December/087729.html
?

Could you review it/integrate changes from it?

> 
> // Robert
> 
> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
> 
>   bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
> 
> are available in the git repository at:
> 
>   git://git.pokylinux.org/poky-contrib rbt/archiver
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
> 
> Robert Yang (5):
>   classes/archive*.bbclass: remove archive-*-source.bbclass
>   archiver.bbclass: refactor it
>   package_rpm.bbclass: archive the source to srpm package
>   archiver.bbclass: move a few code to copyleft_compliance.bbclass
>   local.conf.sample.extended: update for the archiver
> 
>  meta-yocto/conf/local.conf.sample.extended     |  72 +--
>  meta/classes/archive-configured-source.bbclass |  65 ---
>  meta/classes/archive-original-source.bbclass   |  65 ---
>  meta/classes/archive-patched-source.bbclass    |  65 ---
>  meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>  meta/classes/copyleft_compliance.bbclass       |  55 +-
>  meta/classes/package_rpm.bbclass               |  31 +-
>  7 files changed, 316 insertions(+), 760 deletions(-)
>  delete mode 100644 meta/classes/archive-configured-source.bbclass
>  delete mode 100644 meta/classes/archive-original-source.bbclass
>  delete mode 100644 meta/classes/archive-patched-source.bbclass
> 
> -- 
> 1.8.3.1
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Robert Yang - Jan. 13, 2014, 10:47 a.m.
On 01/13/2014 06:21 PM, Martin Jansa wrote:
> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>> * The archive*.bbclass didn't work, and there were a few problems, for
>>    example:
>>    1) There were a few duplicated code
>>    2) There was no src_dir.org (or orig), but the diff command still use
>>       it, and it is not easy to fix this issue if we don't change a lot
>>       of the code.
>>    3) It didn't archive the source for the native or gcc
>>    4) The work flow is not very well
>>    5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>    6) And others ...
>>
>> * So that we have to refactor it, the benefits are:
>>    1) Fix the problems and make it work well.
>>    2) Reduce more than 400 lines in total.
>>    3) Make it easy to use.
>
> Have you seen
> http://lists.openembedded.org/pipermail/openembedded-core/2013-December/087729.html
> ?
>
> Could you review it/integrate changes from it?

I've looked at it just now, I think the problems that you mentioned have been
fixed during the refactor. It won't install/remove the files manually any more,
they will be installed/removed by the sstate. And the usage become easier, just
this would be OK by default:

INHERIT += "archiver"

And we can easily add other functions when needed.

// Robert

>
>>
>> // Robert
>>
>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>
>>    bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>>
>> are available in the git repository at:
>>
>>    git://git.pokylinux.org/poky-contrib rbt/archiver
>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>>
>> Robert Yang (5):
>>    classes/archive*.bbclass: remove archive-*-source.bbclass
>>    archiver.bbclass: refactor it
>>    package_rpm.bbclass: archive the source to srpm package
>>    archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>    local.conf.sample.extended: update for the archiver
>>
>>   meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>   meta/classes/archive-configured-source.bbclass |  65 ---
>>   meta/classes/archive-original-source.bbclass   |  65 ---
>>   meta/classes/archive-patched-source.bbclass    |  65 ---
>>   meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>>   meta/classes/copyleft_compliance.bbclass       |  55 +-
>>   meta/classes/package_rpm.bbclass               |  31 +-
>>   7 files changed, 316 insertions(+), 760 deletions(-)
>>   delete mode 100644 meta/classes/archive-configured-source.bbclass
>>   delete mode 100644 meta/classes/archive-original-source.bbclass
>>   delete mode 100644 meta/classes/archive-patched-source.bbclass
>>
>> --
>> 1.8.3.1
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
Henning Heinold - Jan. 13, 2014, 11:52 a.m.
On Mon, Jan 13, 2014 at 06:47:39PM +0800, Robert Yang wrote:
> 
> On 01/13/2014 06:21 PM, Martin Jansa wrote:
> >On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> >>* The archive*.bbclass didn't work, and there were a few problems, for
> >>   example:
> >>   1) There were a few duplicated code
> >>   2) There was no src_dir.org (or orig), but the diff command still use
> >>      it, and it is not easy to fix this issue if we don't change a lot
> >>      of the code.
> >>   3) It didn't archive the source for the native or gcc
> >>   4) The work flow is not very well
> >>   5) The "subprocess.call('fakeroot cp xxxx'" should be removed
> >>   6) And others ...
> >>
> >>* So that we have to refactor it, the benefits are:
> >>   1) Fix the problems and make it work well.
> >>   2) Reduce more than 400 lines in total.
> >>   3) Make it easy to use.
> >
> >Have you seen
> >http://lists.openembedded.org/pipermail/openembedded-core/2013-December/087729.html
> >?
> >
> >Could you review it/integrate changes from it?
> 
> I've looked at it just now, I think the problems that you mentioned have been
> fixed during the refactor. It won't install/remove the files manually any more,
> they will be installed/removed by the sstate. And the usage become easier, just
> this would be OK by default:
> 
> INHERIT += "archiver"
> 
> And we can easily add other functions when needed.
> 
> // Robert
> 

Hi Robert,

does the whole thread means it is broken in dora too? Are there plans to backport the patches?

Bye Henning
Martin Jansa - Jan. 13, 2014, 11:58 a.m.
On Mon, Jan 13, 2014 at 12:52:49PM +0100, Henning Heinold wrote:
> On Mon, Jan 13, 2014 at 06:47:39PM +0800, Robert Yang wrote:
> > 
> > On 01/13/2014 06:21 PM, Martin Jansa wrote:
> > >On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> > >>* The archive*.bbclass didn't work, and there were a few problems, for
> > >>   example:
> > >>   1) There were a few duplicated code
> > >>   2) There was no src_dir.org (or orig), but the diff command still use
> > >>      it, and it is not easy to fix this issue if we don't change a lot
> > >>      of the code.
> > >>   3) It didn't archive the source for the native or gcc
> > >>   4) The work flow is not very well
> > >>   5) The "subprocess.call('fakeroot cp xxxx'" should be removed
> > >>   6) And others ...
> > >>
> > >>* So that we have to refactor it, the benefits are:
> > >>   1) Fix the problems and make it work well.
> > >>   2) Reduce more than 400 lines in total.
> > >>   3) Make it easy to use.
> > >
> > >Have you seen
> > >http://lists.openembedded.org/pipermail/openembedded-core/2013-December/087729.html
> > >?
> > >
> > >Could you review it/integrate changes from it?
> > 
> > I've looked at it just now, I think the problems that you mentioned have been
> > fixed during the refactor. It won't install/remove the files manually any more,
> > they will be installed/removed by the sstate. And the usage become easier, just
> > this would be OK by default:
> > 
> > INHERIT += "archiver"
> > 
> > And we can easily add other functions when needed.
> > 
> > // Robert
> > 
> 
> Hi Robert,
> 
> does the whole thread means it is broken in dora too? Are there plans to backport the patches?

AFAIK it's broken everywhere, I would like to have it backported to
dylan as well (after testing that it indeed fixes source files being
removed)
Robert Yang - Jan. 13, 2014, 4:20 p.m.
* The archive*.bbclass didn't work, and there were a few problems, for
  example:
  1) There were a few duplicated code
  2) There was no src_dir.org (or orig), but the diff command still use
     it, and it is not easy to fix this issue if we don't change a lot
     of the code.
  3) It didn't archive the source for the native or gcc
  4) The work flow is not very well
  5) The "subprocess.call('fakeroot cp xxxx'" should be removed
  6) And others ...

* So that we have to refactor it, the benefits are:
  1) Fix the problems and make it work well.
  2) Reduce more than 400 lines in total.
  3) Make it easy to use.

// Robert

The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:

  bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib rbt/archiver
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver

Robert Yang (5):
  classes/archive*.bbclass: remove archive-*-source.bbclass
  archiver.bbclass: refactor it
  package_rpm.bbclass: archive the source to srpm package
  archiver.bbclass: move a few code to copyleft_compliance.bbclass
  local.conf.sample.extended: update for the archiver

 meta-yocto/conf/local.conf.sample.extended     |  72 +--
 meta/classes/archive-configured-source.bbclass |  65 ---
 meta/classes/archive-original-source.bbclass   |  65 ---
 meta/classes/archive-patched-source.bbclass    |  65 ---
 meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
 meta/classes/copyleft_compliance.bbclass       |  55 +-
 meta/classes/package_rpm.bbclass               |  31 +-
 7 files changed, 316 insertions(+), 760 deletions(-)
 delete mode 100644 meta/classes/archive-configured-source.bbclass
 delete mode 100644 meta/classes/archive-original-source.bbclass
 delete mode 100644 meta/classes/archive-patched-source.bbclass
Robert Yang - Jan. 14, 2014, 2:13 a.m.
On 01/13/2014 07:58 PM, Martin Jansa wrote:
> On Mon, Jan 13, 2014 at 12:52:49PM +0100, Henning Heinold wrote:
>> On Mon, Jan 13, 2014 at 06:47:39PM +0800, Robert Yang wrote:
>>>
>>> On 01/13/2014 06:21 PM, Martin Jansa wrote:
>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
>>>>>    example:
>>>>>    1) There were a few duplicated code
>>>>>    2) There was no src_dir.org (or orig), but the diff command still use
>>>>>       it, and it is not easy to fix this issue if we don't change a lot
>>>>>       of the code.
>>>>>    3) It didn't archive the source for the native or gcc
>>>>>    4) The work flow is not very well
>>>>>    5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>>>>    6) And others ...
>>>>>
>>>>> * So that we have to refactor it, the benefits are:
>>>>>    1) Fix the problems and make it work well.
>>>>>    2) Reduce more than 400 lines in total.
>>>>>    3) Make it easy to use.
>>>>
>>>> Have you seen
>>>> http://lists.openembedded.org/pipermail/openembedded-core/2013-December/087729.html
>>>> ?
>>>>
>>>> Could you review it/integrate changes from it?
>>>
>>> I've looked at it just now, I think the problems that you mentioned have been
>>> fixed during the refactor. It won't install/remove the files manually any more,
>>> they will be installed/removed by the sstate. And the usage become easier, just
>>> this would be OK by default:
>>>
>>> INHERIT += "archiver"
>>>
>>> And we can easily add other functions when needed.
>>>
>>> // Robert
>>>
>>
>> Hi Robert,
>>
>> does the whole thread means it is broken in dora too? Are there plans to backport the patches?
>
> AFAIK it's broken everywhere, I would like to have it backported to
> dylan as well (after testing that it indeed fixes source files being
> removed)
>

Yes, Martin is right, but I'm not sure whether we should backport them
since we have change a lot of the code, and there should be bugs, usually,
though, I've done as many testing as I can.

BTW., I've updated the code a little in the PULL:

Use "1" or "0" rather than "yes" or "no", which seems more popular in oe,
for example:

ARCHIVER_MODE[dumpdata] = "1" (it was "yes" before).

// Robert


>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
Martin Jansa - Jan. 15, 2014, 4:55 p.m.
On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> * The archive*.bbclass didn't work, and there were a few problems, for
>   example:
>   1) There were a few duplicated code
>   2) There was no src_dir.org (or orig), but the diff command still use
>      it, and it is not easy to fix this issue if we don't change a lot
>      of the code.
>   3) It didn't archive the source for the native or gcc
>   4) The work flow is not very well
>   5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>   6) And others ...
> 
> * So that we have to refactor it, the benefits are:
>   1) Fix the problems and make it work well.
>   2) Reduce more than 400 lines in total.
>   3) Make it easy to use.
> 
> // Robert
> 
> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
> 
>   bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
> 
> are available in the git repository at:
> 
>   git://git.pokylinux.org/poky-contrib rbt/archiver
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver

Can you send pull-request for oe-core?

> Robert Yang (5):
>   classes/archive*.bbclass: remove archive-*-source.bbclass
>   archiver.bbclass: refactor it
>   package_rpm.bbclass: archive the source to srpm package
>   archiver.bbclass: move a few code to copyleft_compliance.bbclass
>   local.conf.sample.extended: update for the archiver
> 
>  meta-yocto/conf/local.conf.sample.extended     |  72 +--
>  meta/classes/archive-configured-source.bbclass |  65 ---
>  meta/classes/archive-original-source.bbclass   |  65 ---
>  meta/classes/archive-patched-source.bbclass    |  65 ---
>  meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>  meta/classes/copyleft_compliance.bbclass       |  55 +-
>  meta/classes/package_rpm.bbclass               |  31 +-
>  7 files changed, 316 insertions(+), 760 deletions(-)
>  delete mode 100644 meta/classes/archive-configured-source.bbclass
>  delete mode 100644 meta/classes/archive-original-source.bbclass
>  delete mode 100644 meta/classes/archive-patched-source.bbclass
> 
> -- 
> 1.8.3.1
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Robert Yang - Jan. 16, 2014, 2:14 a.m.
On 01/16/2014 12:55 AM, Martin Jansa wrote:
> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>> * The archive*.bbclass didn't work, and there were a few problems, for
>>    example:
>>    1) There were a few duplicated code
>>    2) There was no src_dir.org (or orig), but the diff command still use
>>       it, and it is not easy to fix this issue if we don't change a lot
>>       of the code.
>>    3) It didn't archive the source for the native or gcc
>>    4) The work flow is not very well
>>    5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>    6) And others ...
>>
>> * So that we have to refactor it, the benefits are:
>>    1) Fix the problems and make it work well.
>>    2) Reduce more than 400 lines in total.
>>    3) Make it easy to use.
>>
>> // Robert
>>
>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>
>>    bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>>
>> are available in the git repository at:
>>
>>    git://git.pokylinux.org/poky-contrib rbt/archiver
>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>
> Can you send pull-request for oe-core?
>

Here it is:
git://git.pokylinux.org/poky-contrib rbt/ar_oe
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe

// Robert

>> Robert Yang (5):
>>    classes/archive*.bbclass: remove archive-*-source.bbclass
>>    archiver.bbclass: refactor it
>>    package_rpm.bbclass: archive the source to srpm package
>>    archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>    local.conf.sample.extended: update for the archiver
>>
>>   meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>   meta/classes/archive-configured-source.bbclass |  65 ---
>>   meta/classes/archive-original-source.bbclass   |  65 ---
>>   meta/classes/archive-patched-source.bbclass    |  65 ---
>>   meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>>   meta/classes/copyleft_compliance.bbclass       |  55 +-
>>   meta/classes/package_rpm.bbclass               |  31 +-
>>   7 files changed, 316 insertions(+), 760 deletions(-)
>>   delete mode 100644 meta/classes/archive-configured-source.bbclass
>>   delete mode 100644 meta/classes/archive-original-source.bbclass
>>   delete mode 100644 meta/classes/archive-patched-source.bbclass
>>
>> --
>> 1.8.3.1
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
Martin Jansa - Jan. 16, 2014, 9:25 a.m.
On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
> 
> 
> On 01/16/2014 12:55 AM, Martin Jansa wrote:
> > On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> >> * The archive*.bbclass didn't work, and there were a few problems, for
> >>    example:
> >>    1) There were a few duplicated code
> >>    2) There was no src_dir.org (or orig), but the diff command still use
> >>       it, and it is not easy to fix this issue if we don't change a lot
> >>       of the code.
> >>    3) It didn't archive the source for the native or gcc
> >>    4) The work flow is not very well
> >>    5) The "subprocess.call('fakeroot cp xxxx'" should be removed
> >>    6) And others ...
> >>
> >> * So that we have to refactor it, the benefits are:
> >>    1) Fix the problems and make it work well.
> >>    2) Reduce more than 400 lines in total.
> >>    3) Make it easy to use.
> >>
> >> // Robert
> >>
> >> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
> >>
> >>    bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
> >>
> >> are available in the git repository at:
> >>
> >>    git://git.pokylinux.org/poky-contrib rbt/archiver
> >>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
> >
> > Can you send pull-request for oe-core?
> >
> 
> Here it is:
> git://git.pokylinux.org/poky-contrib rbt/ar_oe
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe

This still looks like poky-contrib (so it will download whole
poky-contrib when I add it as remote to oe-core repo).

> // Robert
> 
> >> Robert Yang (5):
> >>    classes/archive*.bbclass: remove archive-*-source.bbclass
> >>    archiver.bbclass: refactor it
> >>    package_rpm.bbclass: archive the source to srpm package
> >>    archiver.bbclass: move a few code to copyleft_compliance.bbclass
> >>    local.conf.sample.extended: update for the archiver
> >>
> >>   meta-yocto/conf/local.conf.sample.extended     |  72 +--
> >>   meta/classes/archive-configured-source.bbclass |  65 ---
> >>   meta/classes/archive-original-source.bbclass   |  65 ---
> >>   meta/classes/archive-patched-source.bbclass    |  65 ---
> >>   meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
> >>   meta/classes/copyleft_compliance.bbclass       |  55 +-
> >>   meta/classes/package_rpm.bbclass               |  31 +-
> >>   7 files changed, 316 insertions(+), 760 deletions(-)
> >>   delete mode 100644 meta/classes/archive-configured-source.bbclass
> >>   delete mode 100644 meta/classes/archive-original-source.bbclass
> >>   delete mode 100644 meta/classes/archive-patched-source.bbclass
> >>
> >> --
> >> 1.8.3.1
> >>
> >> _______________________________________________
> >> Openembedded-core mailing list
> >> Openembedded-core@lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
Robert Yang - Jan. 16, 2014, 9:33 a.m.
On 01/16/2014 05:25 PM, Martin Jansa wrote:
> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
>>
>>
>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>>>> * The archive*.bbclass didn't work, and there were a few problems, for
>>>>     example:
>>>>     1) There were a few duplicated code
>>>>     2) There was no src_dir.org (or orig), but the diff command still use
>>>>        it, and it is not easy to fix this issue if we don't change a lot
>>>>        of the code.
>>>>     3) It didn't archive the source for the native or gcc
>>>>     4) The work flow is not very well
>>>>     5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>>>     6) And others ...
>>>>
>>>> * So that we have to refactor it, the benefits are:
>>>>     1) Fix the problems and make it work well.
>>>>     2) Reduce more than 400 lines in total.
>>>>     3) Make it easy to use.
>>>>
>>>> // Robert
>>>>
>>>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>>>
>>>>     bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>>>>
>>>> are available in the git repository at:
>>>>
>>>>     git://git.pokylinux.org/poky-contrib rbt/archiver
>>>>     http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>>>
>>> Can you send pull-request for oe-core?
>>>
>>
>> Here it is:
>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
>
> This still looks like poky-contrib (so it will download whole
> poky-contrib when I add it as remote to oe-core repo).
>

Sorry, I can't create repo on oe-contrib, seems that I lost my key.

How about:

git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe

// Robert

>> // Robert
>>
>>>> Robert Yang (5):
>>>>     classes/archive*.bbclass: remove archive-*-source.bbclass
>>>>     archiver.bbclass: refactor it
>>>>     package_rpm.bbclass: archive the source to srpm package
>>>>     archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>>>     local.conf.sample.extended: update for the archiver
>>>>
>>>>    meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>>>    meta/classes/archive-configured-source.bbclass |  65 ---
>>>>    meta/classes/archive-original-source.bbclass   |  65 ---
>>>>    meta/classes/archive-patched-source.bbclass    |  65 ---
>>>>    meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>>>>    meta/classes/copyleft_compliance.bbclass       |  55 +-
>>>>    meta/classes/package_rpm.bbclass               |  31 +-
>>>>    7 files changed, 316 insertions(+), 760 deletions(-)
>>>>    delete mode 100644 meta/classes/archive-configured-source.bbclass
>>>>    delete mode 100644 meta/classes/archive-original-source.bbclass
>>>>    delete mode 100644 meta/classes/archive-patched-source.bbclass
>>>>
>>>> --
>>>> 1.8.3.1
>>>>
>>>> _______________________________________________
>>>> Openembedded-core mailing list
>>>> Openembedded-core@lists.openembedded.org
>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>
>
Martin Jansa - Jan. 18, 2014, 12:26 p.m.
On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
> 
> 
> On 01/16/2014 05:25 PM, Martin Jansa wrote:
> > On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
> >>
> >>
> >> On 01/16/2014 12:55 AM, Martin Jansa wrote:
> >>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> >>>> * The archive*.bbclass didn't work, and there were a few problems, for
> >>>>     example:
> >>>>     1) There were a few duplicated code
> >>>>     2) There was no src_dir.org (or orig), but the diff command still use
> >>>>        it, and it is not easy to fix this issue if we don't change a lot
> >>>>        of the code.
> >>>>     3) It didn't archive the source for the native or gcc
> >>>>     4) The work flow is not very well
> >>>>     5) The "subprocess.call('fakeroot cp xxxx'" should be removed
> >>>>     6) And others ...
> >>>>
> >>>> * So that we have to refactor it, the benefits are:
> >>>>     1) Fix the problems and make it work well.
> >>>>     2) Reduce more than 400 lines in total.
> >>>>     3) Make it easy to use.
> >>>>
> >>>> // Robert
> >>>>
> >>>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
> >>>>
> >>>>     bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
> >>>>
> >>>> are available in the git repository at:
> >>>>
> >>>>     git://git.pokylinux.org/poky-contrib rbt/archiver
> >>>>     http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
> >>>
> >>> Can you send pull-request for oe-core?
> >>>
> >>
> >> Here it is:
> >> git://git.pokylinux.org/poky-contrib rbt/ar_oe
> >> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
> >
> > This still looks like poky-contrib (so it will download whole
> > poky-contrib when I add it as remote to oe-core repo).
> >
> 
> Sorry, I can't create repo on oe-contrib, seems that I lost my key.

That's sad, you should send new one to khem to replace it, before
someone else finds your old key.

> How about:
> 
> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe

That still fetches whole poky repo I'm not interested in.

So I've downloaded just archiver.bbclass manually and tested it on my
use-case, I can confirm that it doesn't remove required file anymore,
but it seems that there is something terribly broken when used in
combination with rm_work or multimachine builds.

I haven't debugged it yet, but 2nd build (in the same directory) with
this new archiver included almost every recipe failed in do_patch task.

> >> // Robert
> >>
> >>>> Robert Yang (5):
> >>>>     classes/archive*.bbclass: remove archive-*-source.bbclass
> >>>>     archiver.bbclass: refactor it
> >>>>     package_rpm.bbclass: archive the source to srpm package
> >>>>     archiver.bbclass: move a few code to copyleft_compliance.bbclass
> >>>>     local.conf.sample.extended: update for the archiver
> >>>>
> >>>>    meta-yocto/conf/local.conf.sample.extended     |  72 +--
> >>>>    meta/classes/archive-configured-source.bbclass |  65 ---
> >>>>    meta/classes/archive-original-source.bbclass   |  65 ---
> >>>>    meta/classes/archive-patched-source.bbclass    |  65 ---
> >>>>    meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
> >>>>    meta/classes/copyleft_compliance.bbclass       |  55 +-
> >>>>    meta/classes/package_rpm.bbclass               |  31 +-
> >>>>    7 files changed, 316 insertions(+), 760 deletions(-)
> >>>>    delete mode 100644 meta/classes/archive-configured-source.bbclass
> >>>>    delete mode 100644 meta/classes/archive-original-source.bbclass
> >>>>    delete mode 100644 meta/classes/archive-patched-source.bbclass
> >>>>
> >>>> --
> >>>> 1.8.3.1
> >>>>
> >>>> _______________________________________________
> >>>> Openembedded-core mailing list
> >>>> Openembedded-core@lists.openembedded.org
> >>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >>>
> >
Robert Yang - Jan. 21, 2014, 10:40 a.m.
Hi Mark,

I've tested the archiver.bbclass with rm_work.bbclasss again, it works for me,
it seems that you only downloaded the archiver.bbclass was not enough, would
you please try this PULL? It is really on oe-core-contrib now:-) And if the
error occurs, would you please show the configuration and error logs ?

git://git.openembedded.org/openembedded-core-contrib rbt/archiver

// Robert

On 01/18/2014 08:26 PM, Martin Jansa wrote:
> On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
>>
>>
>> On 01/16/2014 05:25 PM, Martin Jansa wrote:
>>> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
>>>>
>>>>
>>>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
>>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
>>>>>>      example:
>>>>>>      1) There were a few duplicated code
>>>>>>      2) There was no src_dir.org (or orig), but the diff command still use
>>>>>>         it, and it is not easy to fix this issue if we don't change a lot
>>>>>>         of the code.
>>>>>>      3) It didn't archive the source for the native or gcc
>>>>>>      4) The work flow is not very well
>>>>>>      5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>>>>>      6) And others ...
>>>>>>
>>>>>> * So that we have to refactor it, the benefits are:
>>>>>>      1) Fix the problems and make it work well.
>>>>>>      2) Reduce more than 400 lines in total.
>>>>>>      3) Make it easy to use.
>>>>>>
>>>>>> // Robert
>>>>>>
>>>>>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>>>>>
>>>>>>      bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>>>>>>
>>>>>> are available in the git repository at:
>>>>>>
>>>>>>      git://git.pokylinux.org/poky-contrib rbt/archiver
>>>>>>      http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>>>>>
>>>>> Can you send pull-request for oe-core?
>>>>>
>>>>
>>>> Here it is:
>>>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
>>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
>>>
>>> This still looks like poky-contrib (so it will download whole
>>> poky-contrib when I add it as remote to oe-core repo).
>>>
>>
>> Sorry, I can't create repo on oe-contrib, seems that I lost my key.
>
> That's sad, you should send new one to khem to replace it, before
> someone else finds your old key.
>
>> How about:
>>
>> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe
>
> That still fetches whole poky repo I'm not interested in.
>
> So I've downloaded just archiver.bbclass manually and tested it on my
> use-case, I can confirm that it doesn't remove required file anymore,
> but it seems that there is something terribly broken when used in
> combination with rm_work or multimachine builds.
>
> I haven't debugged it yet, but 2nd build (in the same directory) with
> this new archiver included almost every recipe failed in do_patch task.
>
>>>> // Robert
>>>>
>>>>>> Robert Yang (5):
>>>>>>      classes/archive*.bbclass: remove archive-*-source.bbclass
>>>>>>      archiver.bbclass: refactor it
>>>>>>      package_rpm.bbclass: archive the source to srpm package
>>>>>>      archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>>>>>      local.conf.sample.extended: update for the archiver
>>>>>>
>>>>>>     meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>>>>>     meta/classes/archive-configured-source.bbclass |  65 ---
>>>>>>     meta/classes/archive-original-source.bbclass   |  65 ---
>>>>>>     meta/classes/archive-patched-source.bbclass    |  65 ---
>>>>>>     meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>>>>>>     meta/classes/copyleft_compliance.bbclass       |  55 +-
>>>>>>     meta/classes/package_rpm.bbclass               |  31 +-
>>>>>>     7 files changed, 316 insertions(+), 760 deletions(-)
>>>>>>     delete mode 100644 meta/classes/archive-configured-source.bbclass
>>>>>>     delete mode 100644 meta/classes/archive-original-source.bbclass
>>>>>>     delete mode 100644 meta/classes/archive-patched-source.bbclass
>>>>>>
>>>>>> --
>>>>>> 1.8.3.1
>>>>>>
>>>>>> _______________________________________________
>>>>>> Openembedded-core mailing list
>>>>>> Openembedded-core@lists.openembedded.org
>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>
>>>
>
Robert Yang - Jan. 21, 2014, 10:41 a.m.
Hi Martin,

I've tested the archiver.bbclass with rm_work.bbclasss again, it works for me,
it seems that you only downloaded the archiver.bbclass was not enough, would
you please try this PULL? It is based on oe-core and on oe-core-contrib now:-)
And if the error occurs, would you please show the configuration and error
logs ?

git://git.openembedded.org/openembedded-core-contrib rbt/archiver

// Robert

On 01/18/2014 08:26 PM, Martin Jansa wrote:
> On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
>>
>>
>> On 01/16/2014 05:25 PM, Martin Jansa wrote:
>>> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
>>>>
>>>>
>>>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
>>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
>>>>>>      example:
>>>>>>      1) There were a few duplicated code
>>>>>>      2) There was no src_dir.org (or orig), but the diff command still use
>>>>>>         it, and it is not easy to fix this issue if we don't change a lot
>>>>>>         of the code.
>>>>>>      3) It didn't archive the source for the native or gcc
>>>>>>      4) The work flow is not very well
>>>>>>      5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>>>>>      6) And others ...
>>>>>>
>>>>>> * So that we have to refactor it, the benefits are:
>>>>>>      1) Fix the problems and make it work well.
>>>>>>      2) Reduce more than 400 lines in total.
>>>>>>      3) Make it easy to use.
>>>>>>
>>>>>> // Robert
>>>>>>
>>>>>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>>>>>
>>>>>>      bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>>>>>>
>>>>>> are available in the git repository at:
>>>>>>
>>>>>>      git://git.pokylinux.org/poky-contrib rbt/archiver
>>>>>>      http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>>>>>
>>>>> Can you send pull-request for oe-core?
>>>>>
>>>>
>>>> Here it is:
>>>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
>>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
>>>
>>> This still looks like poky-contrib (so it will download whole
>>> poky-contrib when I add it as remote to oe-core repo).
>>>
>>
>> Sorry, I can't create repo on oe-contrib, seems that I lost my key.
>
> That's sad, you should send new one to khem to replace it, before
> someone else finds your old key.
>
>> How about:
>>
>> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe
>
> That still fetches whole poky repo I'm not interested in.
>
> So I've downloaded just archiver.bbclass manually and tested it on my
> use-case, I can confirm that it doesn't remove required file anymore,
> but it seems that there is something terribly broken when used in
> combination with rm_work or multimachine builds.
>
> I haven't debugged it yet, but 2nd build (in the same directory) with
> this new archiver included almost every recipe failed in do_patch task.
>
>>>> // Robert
>>>>
>>>>>> Robert Yang (5):
>>>>>>      classes/archive*.bbclass: remove archive-*-source.bbclass
>>>>>>      archiver.bbclass: refactor it
>>>>>>      package_rpm.bbclass: archive the source to srpm package
>>>>>>      archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>>>>>      local.conf.sample.extended: update for the archiver
>>>>>>
>>>>>>     meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>>>>>     meta/classes/archive-configured-source.bbclass |  65 ---
>>>>>>     meta/classes/archive-original-source.bbclass   |  65 ---
>>>>>>     meta/classes/archive-patched-source.bbclass    |  65 ---
>>>>>>     meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>>>>>>     meta/classes/copyleft_compliance.bbclass       |  55 +-
>>>>>>     meta/classes/package_rpm.bbclass               |  31 +-
>>>>>>     7 files changed, 316 insertions(+), 760 deletions(-)
>>>>>>     delete mode 100644 meta/classes/archive-configured-source.bbclass
>>>>>>     delete mode 100644 meta/classes/archive-original-source.bbclass
>>>>>>     delete mode 100644 meta/classes/archive-patched-source.bbclass
>>>>>>
>>>>>> --
>>>>>> 1.8.3.1
>>>>>>
>>>>>> _______________________________________________
>>>>>> Openembedded-core mailing list
>>>>>> Openembedded-core@lists.openembedded.org
>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>
>>>
>
Martin Jansa - Jan. 21, 2014, 12:37 p.m.
On Tue, Jan 21, 2014 at 06:41:04PM +0800, Robert Yang wrote:
> 
> Hi Martin,
> 
> I've tested the archiver.bbclass with rm_work.bbclasss again, it works for me,
> it seems that you only downloaded the archiver.bbclass was not enough, would
> you please try this PULL? It is based on oe-core and on oe-core-contrib now:-)
> And if the error occurs, would you please show the configuration and error
> logs ?

I've tried to enable "everything" to test it properly, so my config
looks like this (BTW: it would be nice to show in local.conf.sample
what are the default values):

# Create source archive for complience with the licenses
# Enable all to properly test newer archiver backported from oe-core/master
INHERIT_DISTRO += "archiver"
ARCHIVER_MODE[src] = "prepatch"
ARCHIVER_MODE[diff] = "1"
ARCHIVER_MODE[diff-exclude] ?= ".pc autom4te.cache patches"
ARCHIVER_MODE[dumpdata] = "1"
ARCHIVER_MODE[recipe] = "1"
ARCHIVER_MODE[log] = "1"

And error in do_patch:

NOTE: recipe perf-1.0-r8: task do_patch: Started
ERROR: Error executing a python function in /OE/openembedded-core/meta/recipes-kernel/perf/perf.bb:
OSError: [Errno 2] No such file or directory: '/OE/tmp-eglibc/sysroots/qemux86/usr/src'

ERROR: The stack trace of python calls that resulted in this exception/failure was:
ERROR:   File "do_ar_prepatch", line 6, in <module>
ERROR:
ERROR:   File "do_ar_prepatch", line 3, in do_ar_prepatch
ERROR:
ERROR:   File "archiver.bbclass", line 17, in create_tarball
ERROR:
ERROR: The code that was being executed was:
ERROR:      0002:def do_ar_prepatch(d):
ERROR:      0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
ERROR:      0004:
ERROR:      0005:
ERROR:  *** 0006:do_ar_prepatch(d)
ERROR:      0007:
ERROR: [From file: 'do_ar_prepatch', lineno: 6, function: <module>]
ERROR:      0001:
ERROR:      0002:def do_ar_prepatch(d):
ERROR:  *** 0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
ERROR:      0004:
ERROR:      0005:
ERROR:      0006:do_ar_prepatch(d)
ERROR:      0007:
ERROR: [From file: 'do_ar_prepatch', lineno: 3, function: do_ar_prepatch]
ERROR:      0013:    srcdir = srcdir.rstrip('/')
ERROR:      0014:    dirname = os.path.dirname(srcdir)
ERROR:      0015:    basename = os.path.basename(srcdir)
ERROR:      0016:    olddir = os.getcwd()
ERROR:  *** 0017:    os.chdir(dirname)
ERROR:      0018:    bb.note("Creating %s" % tarname)
ERROR:      0019:    if (len(os.listdir(basename))) != 0:
ERROR:      0020:        tar = tarfile.open(tarname, "w:gz")
ERROR:      0021:        tar.add(basename)
ERROR: [From file: 'archiver.bbclass', lineno: 17, function: create_tarball]
ERROR: Function failed: do_ar_prepatch
ERROR: Logfile of failure stored in: /OE/tmp-eglibc/work/i586-oe-linux/perf/1.0-r8/temp/log.do_patch.28364
NOTE: recipe perf-1.0-r8: task do_patch: Failed
ERROR: Task 20159 (/OE/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_patch) failed with exit code '1'

> git://git.openembedded.org/openembedded-core-contrib rbt/archiver
> 
> // Robert
> 
> On 01/18/2014 08:26 PM, Martin Jansa wrote:
> > On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
> >>
> >>
> >> On 01/16/2014 05:25 PM, Martin Jansa wrote:
> >>> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
> >>>>
> >>>>
> >>>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
> >>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> >>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
> >>>>>>      example:
> >>>>>>      1) There were a few duplicated code
> >>>>>>      2) There was no src_dir.org (or orig), but the diff command still use
> >>>>>>         it, and it is not easy to fix this issue if we don't change a lot
> >>>>>>         of the code.
> >>>>>>      3) It didn't archive the source for the native or gcc
> >>>>>>      4) The work flow is not very well
> >>>>>>      5) The "subprocess.call('fakeroot cp xxxx'" should be removed
> >>>>>>      6) And others ...
> >>>>>>
> >>>>>> * So that we have to refactor it, the benefits are:
> >>>>>>      1) Fix the problems and make it work well.
> >>>>>>      2) Reduce more than 400 lines in total.
> >>>>>>      3) Make it easy to use.
> >>>>>>
> >>>>>> // Robert
> >>>>>>
> >>>>>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
> >>>>>>
> >>>>>>      bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
> >>>>>>
> >>>>>> are available in the git repository at:
> >>>>>>
> >>>>>>      git://git.pokylinux.org/poky-contrib rbt/archiver
> >>>>>>      http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
> >>>>>
> >>>>> Can you send pull-request for oe-core?
> >>>>>
> >>>>
> >>>> Here it is:
> >>>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
> >>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
> >>>
> >>> This still looks like poky-contrib (so it will download whole
> >>> poky-contrib when I add it as remote to oe-core repo).
> >>>
> >>
> >> Sorry, I can't create repo on oe-contrib, seems that I lost my key.
> >
> > That's sad, you should send new one to khem to replace it, before
> > someone else finds your old key.
> >
> >> How about:
> >>
> >> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe
> >
> > That still fetches whole poky repo I'm not interested in.
> >
> > So I've downloaded just archiver.bbclass manually and tested it on my
> > use-case, I can confirm that it doesn't remove required file anymore,
> > but it seems that there is something terribly broken when used in
> > combination with rm_work or multimachine builds.
> >
> > I haven't debugged it yet, but 2nd build (in the same directory) with
> > this new archiver included almost every recipe failed in do_patch task.
> >
> >>>> // Robert
> >>>>
> >>>>>> Robert Yang (5):
> >>>>>>      classes/archive*.bbclass: remove archive-*-source.bbclass
> >>>>>>      archiver.bbclass: refactor it
> >>>>>>      package_rpm.bbclass: archive the source to srpm package
> >>>>>>      archiver.bbclass: move a few code to copyleft_compliance.bbclass
> >>>>>>      local.conf.sample.extended: update for the archiver
> >>>>>>
> >>>>>>     meta-yocto/conf/local.conf.sample.extended     |  72 +--
> >>>>>>     meta/classes/archive-configured-source.bbclass |  65 ---
> >>>>>>     meta/classes/archive-original-source.bbclass   |  65 ---
> >>>>>>     meta/classes/archive-patched-source.bbclass    |  65 ---
> >>>>>>     meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
> >>>>>>     meta/classes/copyleft_compliance.bbclass       |  55 +-
> >>>>>>     meta/classes/package_rpm.bbclass               |  31 +-
> >>>>>>     7 files changed, 316 insertions(+), 760 deletions(-)
> >>>>>>     delete mode 100644 meta/classes/archive-configured-source.bbclass
> >>>>>>     delete mode 100644 meta/classes/archive-original-source.bbclass
> >>>>>>     delete mode 100644 meta/classes/archive-patched-source.bbclass
> >>>>>>
> >>>>>> --
> >>>>>> 1.8.3.1
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Openembedded-core mailing list
> >>>>>> Openembedded-core@lists.openembedded.org
> >>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >>>>>
> >>>
> >
Robert Yang - Jan. 22, 2014, 2:59 a.m.
On 01/21/2014 08:37 PM, Martin Jansa wrote:
> On Tue, Jan 21, 2014 at 06:41:04PM +0800, Robert Yang wrote:
>>
>> Hi Martin,
>>
>> I've tested the archiver.bbclass with rm_work.bbclasss again, it works for me,
>> it seems that you only downloaded the archiver.bbclass was not enough, would
>> you please try this PULL? It is based on oe-core and on oe-core-contrib now:-)
>> And if the error occurs, would you please show the configuration and error
>> logs ?
>
> I've tried to enable "everything" to test it properly, so my config
> looks like this (BTW: it would be nice to show in local.conf.sample
> what are the default values):
>
> # Create source archive for complience with the licenses
> # Enable all to properly test newer archiver backported from oe-core/master
> INHERIT_DISTRO += "archiver"

I changed it to INHERIT += "archiver" since the INHERIT_DISTRO will override
this:

meta/conf/distro/defaultsetup.conf:INHERIT_DISTRO ?= "debian devshell sstate 
license"

Everything works fine to me, I'm afraid that set INHERIT_DISTRO += "archiver"
may override other INHERIT_DISTRO's settings.

I also added INHERIT += "rm_work"

And please see my comments below.

> ARCHIVER_MODE[src] = "prepatch"
> ARCHIVER_MODE[diff] = "1"
> ARCHIVER_MODE[diff-exclude] ?= ".pc autom4te.cache patches"
> ARCHIVER_MODE[dumpdata] = "1"
> ARCHIVER_MODE[recipe] = "1"
> ARCHIVER_MODE[log] = "1"
>
> And error in do_patch:
>
> NOTE: recipe perf-1.0-r8: task do_patch: Started
> ERROR: Error executing a python function in /OE/openembedded-core/meta/recipes-kernel/perf/perf.bb:
> OSError: [Errno 2] No such file or directory: '/OE/tmp-eglibc/sysroots/qemux86/usr/src'

The perf's ${S} is sysroots/qemux86/usr/src/kernel, but the problem is that why
the /OE/tmp-eglibc/sysroots/qemux86/usr/src doesn't exist ? The rm_work doesn't
remove it, no matter how many times I tried, it still works.

So, please:

1) Check whether your other INHERIT_DISTRO settings have been overrided or not,
    and try to use INHERIT rather than INHERIT_DISTRO ?

2) If it still fails, would you please provide more info ? For example, how to
    reproduce it, exactly ?

3) It would be great if we can talk oneline, how about gtalk ? I will try to
    reach you there.

4) It would be great if others can help me to test it.

// Robert

>
> ERROR: The stack trace of python calls that resulted in this exception/failure was:
> ERROR:   File "do_ar_prepatch", line 6, in <module>
> ERROR:
> ERROR:   File "do_ar_prepatch", line 3, in do_ar_prepatch
> ERROR:
> ERROR:   File "archiver.bbclass", line 17, in create_tarball
> ERROR:
> ERROR: The code that was being executed was:
> ERROR:      0002:def do_ar_prepatch(d):
> ERROR:      0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
> ERROR:      0004:
> ERROR:      0005:
> ERROR:  *** 0006:do_ar_prepatch(d)
> ERROR:      0007:
> ERROR: [From file: 'do_ar_prepatch', lineno: 6, function: <module>]
> ERROR:      0001:
> ERROR:      0002:def do_ar_prepatch(d):
> ERROR:  *** 0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
> ERROR:      0004:
> ERROR:      0005:
> ERROR:      0006:do_ar_prepatch(d)
> ERROR:      0007:
> ERROR: [From file: 'do_ar_prepatch', lineno: 3, function: do_ar_prepatch]
> ERROR:      0013:    srcdir = srcdir.rstrip('/')
> ERROR:      0014:    dirname = os.path.dirname(srcdir)
> ERROR:      0015:    basename = os.path.basename(srcdir)
> ERROR:      0016:    olddir = os.getcwd()
> ERROR:  *** 0017:    os.chdir(dirname)
> ERROR:      0018:    bb.note("Creating %s" % tarname)
> ERROR:      0019:    if (len(os.listdir(basename))) != 0:
> ERROR:      0020:        tar = tarfile.open(tarname, "w:gz")
> ERROR:      0021:        tar.add(basename)
> ERROR: [From file: 'archiver.bbclass', lineno: 17, function: create_tarball]
> ERROR: Function failed: do_ar_prepatch
> ERROR: Logfile of failure stored in: /OE/tmp-eglibc/work/i586-oe-linux/perf/1.0-r8/temp/log.do_patch.28364
> NOTE: recipe perf-1.0-r8: task do_patch: Failed
> ERROR: Task 20159 (/OE/openembedded-core/meta/recipes-kernel/perf/perf.bb, do_patch) failed with exit code '1'
>
>> git://git.openembedded.org/openembedded-core-contrib rbt/archiver
>>
>> // Robert
>>
>> On 01/18/2014 08:26 PM, Martin Jansa wrote:
>>> On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
>>>>
>>>>
>>>> On 01/16/2014 05:25 PM, Martin Jansa wrote:
>>>>> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
>>>>>>
>>>>>>
>>>>>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
>>>>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>>>>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
>>>>>>>>       example:
>>>>>>>>       1) There were a few duplicated code
>>>>>>>>       2) There was no src_dir.org (or orig), but the diff command still use
>>>>>>>>          it, and it is not easy to fix this issue if we don't change a lot
>>>>>>>>          of the code.
>>>>>>>>       3) It didn't archive the source for the native or gcc
>>>>>>>>       4) The work flow is not very well
>>>>>>>>       5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>>>>>>>       6) And others ...
>>>>>>>>
>>>>>>>> * So that we have to refactor it, the benefits are:
>>>>>>>>       1) Fix the problems and make it work well.
>>>>>>>>       2) Reduce more than 400 lines in total.
>>>>>>>>       3) Make it easy to use.
>>>>>>>>
>>>>>>>> // Robert
>>>>>>>>
>>>>>>>> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>>>>>>>
>>>>>>>>       bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>>>>>>>>
>>>>>>>> are available in the git repository at:
>>>>>>>>
>>>>>>>>       git://git.pokylinux.org/poky-contrib rbt/archiver
>>>>>>>>       http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>>>>>>>
>>>>>>> Can you send pull-request for oe-core?
>>>>>>>
>>>>>>
>>>>>> Here it is:
>>>>>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
>>>>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
>>>>>
>>>>> This still looks like poky-contrib (so it will download whole
>>>>> poky-contrib when I add it as remote to oe-core repo).
>>>>>
>>>>
>>>> Sorry, I can't create repo on oe-contrib, seems that I lost my key.
>>>
>>> That's sad, you should send new one to khem to replace it, before
>>> someone else finds your old key.
>>>
>>>> How about:
>>>>
>>>> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe
>>>
>>> That still fetches whole poky repo I'm not interested in.
>>>
>>> So I've downloaded just archiver.bbclass manually and tested it on my
>>> use-case, I can confirm that it doesn't remove required file anymore,
>>> but it seems that there is something terribly broken when used in
>>> combination with rm_work or multimachine builds.
>>>
>>> I haven't debugged it yet, but 2nd build (in the same directory) with
>>> this new archiver included almost every recipe failed in do_patch task.
>>>
>>>>>> // Robert
>>>>>>
>>>>>>>> Robert Yang (5):
>>>>>>>>       classes/archive*.bbclass: remove archive-*-source.bbclass
>>>>>>>>       archiver.bbclass: refactor it
>>>>>>>>       package_rpm.bbclass: archive the source to srpm package
>>>>>>>>       archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>>>>>>>       local.conf.sample.extended: update for the archiver
>>>>>>>>
>>>>>>>>      meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>>>>>>>      meta/classes/archive-configured-source.bbclass |  65 ---
>>>>>>>>      meta/classes/archive-original-source.bbclass   |  65 ---
>>>>>>>>      meta/classes/archive-patched-source.bbclass    |  65 ---
>>>>>>>>      meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>>>>>>>>      meta/classes/copyleft_compliance.bbclass       |  55 +-
>>>>>>>>      meta/classes/package_rpm.bbclass               |  31 +-
>>>>>>>>      7 files changed, 316 insertions(+), 760 deletions(-)
>>>>>>>>      delete mode 100644 meta/classes/archive-configured-source.bbclass
>>>>>>>>      delete mode 100644 meta/classes/archive-original-source.bbclass
>>>>>>>>      delete mode 100644 meta/classes/archive-patched-source.bbclass
>>>>>>>>
>>>>>>>> --
>>>>>>>> 1.8.3.1
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Openembedded-core mailing list
>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>>>
>>>>>
>>>
>
Robert Yang - Jan. 22, 2014, 10:13 a.m.
On 01/22/2014 10:59 AM, Robert Yang wrote:
>
>
> On 01/21/2014 08:37 PM, Martin Jansa wrote:
>> On Tue, Jan 21, 2014 at 06:41:04PM +0800, Robert Yang wrote:
>>>
>>> Hi Martin,
>>>

I've talked with Martin, the difference is that he tried it on dylan,
this is only for master, I'd like to backport if for dora, we may also
backport it for dylan, but this may need more discussions.

// Robert


>>> I've tested the archiver.bbclass with rm_work.bbclasss again, it works for me,
>>> it seems that you only downloaded the archiver.bbclass was not enough, would
>>> you please try this PULL? It is based on oe-core and on oe-core-contrib now:-)
>>> And if the error occurs, would you please show the configuration and error
>>> logs ?
>>
>> I've tried to enable "everything" to test it properly, so my config
>> looks like this (BTW: it would be nice to show in local.conf.sample
>> what are the default values):
>>
>> # Create source archive for complience with the licenses
>> # Enable all to properly test newer archiver backported from oe-core/master
>> INHERIT_DISTRO += "archiver"
>
> I changed it to INHERIT += "archiver" since the INHERIT_DISTRO will override
> this:
>
> meta/conf/distro/defaultsetup.conf:INHERIT_DISTRO ?= "debian devshell sstate
> license"
>
> Everything works fine to me, I'm afraid that set INHERIT_DISTRO += "archiver"
> may override other INHERIT_DISTRO's settings.
>
> I also added INHERIT += "rm_work"
>
> And please see my comments below.
>
>> ARCHIVER_MODE[src] = "prepatch"
>> ARCHIVER_MODE[diff] = "1"
>> ARCHIVER_MODE[diff-exclude] ?= ".pc autom4te.cache patches"
>> ARCHIVER_MODE[dumpdata] = "1"
>> ARCHIVER_MODE[recipe] = "1"
>> ARCHIVER_MODE[log] = "1"
>>
>> And error in do_patch:
>>
>> NOTE: recipe perf-1.0-r8: task do_patch: Started
>> ERROR: Error executing a python function in
>> /OE/openembedded-core/meta/recipes-kernel/perf/perf.bb:
>> OSError: [Errno 2] No such file or directory:
>> '/OE/tmp-eglibc/sysroots/qemux86/usr/src'
>
> The perf's ${S} is sysroots/qemux86/usr/src/kernel, but the problem is that why
> the /OE/tmp-eglibc/sysroots/qemux86/usr/src doesn't exist ? The rm_work doesn't
> remove it, no matter how many times I tried, it still works.
>
> So, please:
>
> 1) Check whether your other INHERIT_DISTRO settings have been overrided or not,
>     and try to use INHERIT rather than INHERIT_DISTRO ?
>
> 2) If it still fails, would you please provide more info ? For example, how to
>     reproduce it, exactly ?
>
> 3) It would be great if we can talk oneline, how about gtalk ? I will try to
>     reach you there.
>
> 4) It would be great if others can help me to test it.
>
> // Robert
>
>>
>> ERROR: The stack trace of python calls that resulted in this exception/failure
>> was:
>> ERROR:   File "do_ar_prepatch", line 6, in <module>
>> ERROR:
>> ERROR:   File "do_ar_prepatch", line 3, in do_ar_prepatch
>> ERROR:
>> ERROR:   File "archiver.bbclass", line 17, in create_tarball
>> ERROR:
>> ERROR: The code that was being executed was:
>> ERROR:      0002:def do_ar_prepatch(d):
>> ERROR:      0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
>> ERROR:      0004:
>> ERROR:      0005:
>> ERROR:  *** 0006:do_ar_prepatch(d)
>> ERROR:      0007:
>> ERROR: [From file: 'do_ar_prepatch', lineno: 6, function: <module>]
>> ERROR:      0001:
>> ERROR:      0002:def do_ar_prepatch(d):
>> ERROR:  *** 0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
>> ERROR:      0004:
>> ERROR:      0005:
>> ERROR:      0006:do_ar_prepatch(d)
>> ERROR:      0007:
>> ERROR: [From file: 'do_ar_prepatch', lineno: 3, function: do_ar_prepatch]
>> ERROR:      0013:    srcdir = srcdir.rstrip('/')
>> ERROR:      0014:    dirname = os.path.dirname(srcdir)
>> ERROR:      0015:    basename = os.path.basename(srcdir)
>> ERROR:      0016:    olddir = os.getcwd()
>> ERROR:  *** 0017:    os.chdir(dirname)
>> ERROR:      0018:    bb.note("Creating %s" % tarname)
>> ERROR:      0019:    if (len(os.listdir(basename))) != 0:
>> ERROR:      0020:        tar = tarfile.open(tarname, "w:gz")
>> ERROR:      0021:        tar.add(basename)
>> ERROR: [From file: 'archiver.bbclass', lineno: 17, function: create_tarball]
>> ERROR: Function failed: do_ar_prepatch
>> ERROR: Logfile of failure stored in:
>> /OE/tmp-eglibc/work/i586-oe-linux/perf/1.0-r8/temp/log.do_patch.28364
>> NOTE: recipe perf-1.0-r8: task do_patch: Failed
>> ERROR: Task 20159 (/OE/openembedded-core/meta/recipes-kernel/perf/perf.bb,
>> do_patch) failed with exit code '1'
>>
>>> git://git.openembedded.org/openembedded-core-contrib rbt/archiver
>>>
>>> // Robert
>>>
>>> On 01/18/2014 08:26 PM, Martin Jansa wrote:
>>>> On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
>>>>>
>>>>>
>>>>> On 01/16/2014 05:25 PM, Martin Jansa wrote:
>>>>>> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
>>>>>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
>>>>>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
>>>>>>>>>       example:
>>>>>>>>>       1) There were a few duplicated code
>>>>>>>>>       2) There was no src_dir.org (or orig), but the diff command still
>>>>>>>>> use
>>>>>>>>>          it, and it is not easy to fix this issue if we don't change a lot
>>>>>>>>>          of the code.
>>>>>>>>>       3) It didn't archive the source for the native or gcc
>>>>>>>>>       4) The work flow is not very well
>>>>>>>>>       5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>>>>>>>>>       6) And others ...
>>>>>>>>>
>>>>>>>>> * So that we have to refactor it, the benefits are:
>>>>>>>>>       1) Fix the problems and make it work well.
>>>>>>>>>       2) Reduce more than 400 lines in total.
>>>>>>>>>       3) Make it easy to use.
>>>>>>>>>
>>>>>>>>> // Robert
>>>>>>>>>
>>>>>>>>> The following changes since commit
>>>>>>>>> 1b636173ca88e5ccca1992f9a12367a1189fa674:
>>>>>>>>>
>>>>>>>>>       bitbake: toaster: Toaster GUI, generic search, filter and order
>>>>>>>>> (2014-01-10 15:20:26 +0000)
>>>>>>>>>
>>>>>>>>> are available in the git repository at:
>>>>>>>>>
>>>>>>>>>       git://git.pokylinux.org/poky-contrib rbt/archiver
>>>>>>>>>       http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>>>>>>>>
>>>>>>>> Can you send pull-request for oe-core?
>>>>>>>>
>>>>>>>
>>>>>>> Here it is:
>>>>>>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
>>>>>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
>>>>>>
>>>>>> This still looks like poky-contrib (so it will download whole
>>>>>> poky-contrib when I add it as remote to oe-core repo).
>>>>>>
>>>>>
>>>>> Sorry, I can't create repo on oe-contrib, seems that I lost my key.
>>>>
>>>> That's sad, you should send new one to khem to replace it, before
>>>> someone else finds your old key.
>>>>
>>>>> How about:
>>>>>
>>>>> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe
>>>>
>>>> That still fetches whole poky repo I'm not interested in.
>>>>
>>>> So I've downloaded just archiver.bbclass manually and tested it on my
>>>> use-case, I can confirm that it doesn't remove required file anymore,
>>>> but it seems that there is something terribly broken when used in
>>>> combination with rm_work or multimachine builds.
>>>>
>>>> I haven't debugged it yet, but 2nd build (in the same directory) with
>>>> this new archiver included almost every recipe failed in do_patch task.
>>>>
>>>>>>> // Robert
>>>>>>>
>>>>>>>>> Robert Yang (5):
>>>>>>>>>       classes/archive*.bbclass: remove archive-*-source.bbclass
>>>>>>>>>       archiver.bbclass: refactor it
>>>>>>>>>       package_rpm.bbclass: archive the source to srpm package
>>>>>>>>>       archiver.bbclass: move a few code to copyleft_compliance.bbclass
>>>>>>>>>       local.conf.sample.extended: update for the archiver
>>>>>>>>>
>>>>>>>>>      meta-yocto/conf/local.conf.sample.extended     |  72 +--
>>>>>>>>>      meta/classes/archive-configured-source.bbclass |  65 ---
>>>>>>>>>      meta/classes/archive-original-source.bbclass   |  65 ---
>>>>>>>>>      meta/classes/archive-patched-source.bbclass    |  65 ---
>>>>>>>>>      meta/classes/archiver.bbclass                  | 723
>>>>>>>>> ++++++++-----------------
>>>>>>>>>      meta/classes/copyleft_compliance.bbclass       |  55 +-
>>>>>>>>>      meta/classes/package_rpm.bbclass               |  31 +-
>>>>>>>>>      7 files changed, 316 insertions(+), 760 deletions(-)
>>>>>>>>>      delete mode 100644 meta/classes/archive-configured-source.bbclass
>>>>>>>>>      delete mode 100644 meta/classes/archive-original-source.bbclass
>>>>>>>>>      delete mode 100644 meta/classes/archive-patched-source.bbclass
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> 1.8.3.1
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Openembedded-core mailing list
>>>>>>>>> Openembedded-core@lists.openembedded.org
>>>>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>>>>>>
>>>>>>
>>>>
>>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
Martin Jansa - Jan. 31, 2014, 3:57 p.m.
On Wed, Jan 22, 2014 at 06:13:37PM +0800, Robert Yang wrote:
> 
> On 01/22/2014 10:59 AM, Robert Yang wrote:
> >
> >
> > On 01/21/2014 08:37 PM, Martin Jansa wrote:
> >> On Tue, Jan 21, 2014 at 06:41:04PM +0800, Robert Yang wrote:
> >>>
> >>> Hi Martin,
> >>>
> 
> I've talked with Martin, the difference is that he tried it on dylan,
> this is only for master, I'd like to backport if for dora, we may also
> backport it for dylan, but this may need more discussions.

Have you tried multi-machine build as suggested? In master I see the
same behavior, these tasks are reexecuted after MACHINE switch causing
not only rebuild of everything (probably caused by combination with
rm_work), but also warnings like this:

NOTE: recipe gcc-cross-4.8.2-r0: task do_deploy_archives: Started
WARNING: The recipe gcc-cross is trying to install files into a shared
area when those files already exist. Those files and their manifest
location are:
   /OE/build/oe-core/tmp-eglibc/deploy/sources/i586-oe-linux/gcc-cross-4.8.2-r0/gcc-cross-4.8.2-r0-log.tar.gz
   Matched in manifest-x86_64_qemux86-gcc-cross.deploy_archives
   /OE/build/oe-core/tmp-eglibc/deploy/sources/i586-oe-linux/gcc-cross-4.8.2-r0/gcc-cross-4.8.2-r0-showdata.dump
   Matched in manifest-x86_64_qemux86-gcc-cross.deploy_archives
   /OE/build/oe-core/tmp-eglibc/deploy/sources/i586-oe-linux/gcc-cross-4.8.2-r0/gcc-cross-4.8.2-r0-recipe.tar.gz
   Matched in manifest-x86_64_qemux86-gcc-cross.deploy_archives
Please verify which package should provide the above files.
NOTE: recipe gcc-cross-4.8.2-r0: task do_deploy_archives: Succeeded

and in some cases enabling archiver (or changing its configuration)
causes not only archiver-related tasks to be executed (do_dumpdata,
do_ar_recipe, do_ar_log, do_deploy_archives) but triggers also another
do_patch task in already patched ${S} (probably because of changed
prefunc signature), causing do_patch errors.

(I haven't found simple reproducer, maybe its related
to cache as well, because repeating the failing setup in the same tmpdir
doesn't work - but also doesn't work when I delete cache and
tmp-eglibc/cache between steps).

Interestingly bitbake -S doesn't show different signatures when compared
with:
oe-core/scripts/sstate-diff-machines.sh --tmpdir=tmp-eglibc --targets="libffi" --machines="qemux86 qemux86copy"

you can cherry-pick qemux86copy test MACHINE from:
http://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/tune2-test

I'm testing it on libffi recipe, because it has extra patch in SRC_URI
and exists also in -native version which is even faster to build.

OE qemux86@ ~/build/oe-core $ bitbake libffi | tee log.qemux86.prepatch
NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 47743, PID: 8217
Parsing recipes...WARNING: Unable to get checksum for cogl-1.0 SRC_URI entry kms-include-stddef.h-before-drm.h.patch: file could not be found
done.
Parsing of 847 .bb files complete (0 cached, 847 parsed). 1203 targets, 35 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.21.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Gentoo-2.2"
TARGET_SYS        = "i586-oe-linux"
MACHINE           = "qemux86"
DISTRO            = "nodistro"
DISTRO_VERSION    = "nodistro.0"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              = "jansa/test:9d7f81bc7257b9e5f902daa11ce63c84046256d2"

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Running setscene task 121 of 128 (/OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_populate_lic_setscene)
NOTE: Running setscene task 122 of 128 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/libtool/libtool-cross_2.4.2.bb, do_populate_lic_setscene)
NOTE: recipe libffi-3.0.13-r0: task do_populate_lic_setscene: Started
NOTE: recipe libtool-cross-2.4.2-r6.1: task do_populate_lic_setscene: Started
....
NOTE: Running task 725 of 728 (ID: 17, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_deploy_archives)
NOTE: recipe libffi-3.0.13-r0: task do_deploy_archives: Started
NOTE: recipe libffi-3.0.13-r0: task do_deploy_archives: Succeeded
NOTE: Running noexec task 726 of 728 (ID: 11, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_build)
NOTE: Running task 727 of 728 (ID: 12, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_rm_work)
NOTE: recipe libffi-3.0.13-r0: task do_rm_work: Started
NOTE: recipe libffi-3.0.13-r0: task do_rm_work: Succeeded
NOTE: Running task 728 of 728 (ID: 13, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_rm_work_all)
NOTE: recipe libffi-3.0.13-r0: task do_rm_work_all: Started
NOTE: recipe libffi-3.0.13-r0: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 728 tasks of which 74 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory

Summary: There was 1 WARNING message shown.
OE qemux86@ ~/build/oe-core $ export MACHINE=qemux86copy
OE qemux86copy@ ~/build/oe-core $ bitbake libffi | tee log.qemux86.prepatch2
NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 55093, PID: 7136
Parsing recipes...WARNING: Unable to get checksum for cogl-1.0 SRC_URI entry kms-include-stddef.h-before-drm.h.patch: file could not be found
done.
Parsing of 847 .bb files complete (0 cached, 847 parsed). 1203 targets, 35 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
 
Build Configuration:
BB_VERSION        = "1.21.1"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Gentoo-2.2"
TARGET_SYS        = "i586-oe-linux"
MACHINE           = "qemux86copy"
DISTRO            = "nodistro"
DISTRO_VERSION    = "nodistro.0"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              = "jansa/test:9d7f81bc7257b9e5f902daa11ce63c84046256d2"
 
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Running setscene task 99 of 128 (/OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_populate_sysroot_setscene)
NOTE: Running setscene task 100 of 128 (/OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_package_write_ipk_setscene)
NOTE: recipe libffi-3.0.13-r0: task do_populate_sysroot_setscene: Started
NOTE: recipe libffi-3.0.13-r0: task do_package_write_ipk_setscene: Started
NOTE: recipe libffi-3.0.13-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: Running setscene task 101 of 128 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/libtool/libtool-cross_2.4.2.bb, do_package_write_ipk_setscene)
...
NOTE: Running task 725 of 728 (ID: 17, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_deploy_archives)
NOTE: recipe libffi-3.0.13-r0: task do_deploy_archives: Started
NOTE: recipe libffi-3.0.13-r0: task do_deploy_archives: Succeeded
NOTE: Running noexec task 726 of 728 (ID: 11, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_build)
NOTE: Running task 727 of 728 (ID: 12, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_rm_work)
NOTE: recipe libffi-3.0.13-r0: task do_rm_work: Started
NOTE: recipe libffi-3.0.13-r0: task do_rm_work: Succeeded
NOTE: Running task 728 of 728 (ID: 13, /OE/build/oe-core/openembedded-core/meta/recipes-gnome/libffi/libffi_3.0.13.bb, do_rm_work_all)
NOTE: recipe libffi-3.0.13-r0: task do_rm_work_all: Started
NOTE: recipe libffi-3.0.13-r0: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 728 tasks of which 110 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory

Summary: There were 4 WARNING messages shown.

Normally this should reuse everything from sstate and build only kernel and very few MACHINE_ARCH recipes.

conf/local.conf:
INHERIT += "rm_work"

INHERIT += "archiver"
ARCHIVER_MODE[src] = "prepatch"
ARCHIVER_MODE[diff] = "1"
ARCHIVER_MODE[diff-exclude] ?= ".pc autom4te.cache patches no yes"
ARCHIVER_MODE[dumpdata] = "1"
ARCHIVER_MODE[recipe] = "1"
ARCHIVER_MODE[log] = "1"

> >>> I've tested the archiver.bbclass with rm_work.bbclasss again, it works for me,
> >>> it seems that you only downloaded the archiver.bbclass was not enough, would
> >>> you please try this PULL? It is based on oe-core and on oe-core-contrib now:-)
> >>> And if the error occurs, would you please show the configuration and error
> >>> logs ?
> >>
> >> I've tried to enable "everything" to test it properly, so my config
> >> looks like this (BTW: it would be nice to show in local.conf.sample
> >> what are the default values):
> >>
> >> # Create source archive for complience with the licenses
> >> # Enable all to properly test newer archiver backported from oe-core/master
> >> INHERIT_DISTRO += "archiver"
> >
> > I changed it to INHERIT += "archiver" since the INHERIT_DISTRO will override
> > this:
> >
> > meta/conf/distro/defaultsetup.conf:INHERIT_DISTRO ?= "debian devshell sstate
> > license"
> >
> > Everything works fine to me, I'm afraid that set INHERIT_DISTRO += "archiver"
> > may override other INHERIT_DISTRO's settings.
> >
> > I also added INHERIT += "rm_work"
> >
> > And please see my comments below.
> >
> >> ARCHIVER_MODE[src] = "prepatch"
> >> ARCHIVER_MODE[diff] = "1"
> >> ARCHIVER_MODE[diff-exclude] ?= ".pc autom4te.cache patches"
> >> ARCHIVER_MODE[dumpdata] = "1"
> >> ARCHIVER_MODE[recipe] = "1"
> >> ARCHIVER_MODE[log] = "1"
> >>
> >> And error in do_patch:
> >>
> >> NOTE: recipe perf-1.0-r8: task do_patch: Started
> >> ERROR: Error executing a python function in
> >> /OE/openembedded-core/meta/recipes-kernel/perf/perf.bb:
> >> OSError: [Errno 2] No such file or directory:
> >> '/OE/tmp-eglibc/sysroots/qemux86/usr/src'
> >
> > The perf's ${S} is sysroots/qemux86/usr/src/kernel, but the problem is that why
> > the /OE/tmp-eglibc/sysroots/qemux86/usr/src doesn't exist ? The rm_work doesn't
> > remove it, no matter how many times I tried, it still works.
> >
> > So, please:
> >
> > 1) Check whether your other INHERIT_DISTRO settings have been overrided or not,
> >     and try to use INHERIT rather than INHERIT_DISTRO ?
> >
> > 2) If it still fails, would you please provide more info ? For example, how to
> >     reproduce it, exactly ?
> >
> > 3) It would be great if we can talk oneline, how about gtalk ? I will try to
> >     reach you there.
> >
> > 4) It would be great if others can help me to test it.
> >
> > // Robert
> >
> >>
> >> ERROR: The stack trace of python calls that resulted in this exception/failure
> >> was:
> >> ERROR:   File "do_ar_prepatch", line 6, in <module>
> >> ERROR:
> >> ERROR:   File "do_ar_prepatch", line 3, in do_ar_prepatch
> >> ERROR:
> >> ERROR:   File "archiver.bbclass", line 17, in create_tarball
> >> ERROR:
> >> ERROR: The code that was being executed was:
> >> ERROR:      0002:def do_ar_prepatch(d):
> >> ERROR:      0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
> >> ERROR:      0004:
> >> ERROR:      0005:
> >> ERROR:  *** 0006:do_ar_prepatch(d)
> >> ERROR:      0007:
> >> ERROR: [From file: 'do_ar_prepatch', lineno: 6, function: <module>]
> >> ERROR:      0001:
> >> ERROR:      0002:def do_ar_prepatch(d):
> >> ERROR:  *** 0003:    create_tarball(d, d.getVar('S', True), 'prepatch')
> >> ERROR:      0004:
> >> ERROR:      0005:
> >> ERROR:      0006:do_ar_prepatch(d)
> >> ERROR:      0007:
> >> ERROR: [From file: 'do_ar_prepatch', lineno: 3, function: do_ar_prepatch]
> >> ERROR:      0013:    srcdir = srcdir.rstrip('/')
> >> ERROR:      0014:    dirname = os.path.dirname(srcdir)
> >> ERROR:      0015:    basename = os.path.basename(srcdir)
> >> ERROR:      0016:    olddir = os.getcwd()
> >> ERROR:  *** 0017:    os.chdir(dirname)
> >> ERROR:      0018:    bb.note("Creating %s" % tarname)
> >> ERROR:      0019:    if (len(os.listdir(basename))) != 0:
> >> ERROR:      0020:        tar = tarfile.open(tarname, "w:gz")
> >> ERROR:      0021:        tar.add(basename)
> >> ERROR: [From file: 'archiver.bbclass', lineno: 17, function: create_tarball]
> >> ERROR: Function failed: do_ar_prepatch
> >> ERROR: Logfile of failure stored in:
> >> /OE/tmp-eglibc/work/i586-oe-linux/perf/1.0-r8/temp/log.do_patch.28364
> >> NOTE: recipe perf-1.0-r8: task do_patch: Failed
> >> ERROR: Task 20159 (/OE/openembedded-core/meta/recipes-kernel/perf/perf.bb,
> >> do_patch) failed with exit code '1'
> >>
> >>> git://git.openembedded.org/openembedded-core-contrib rbt/archiver
> >>>
> >>> // Robert
> >>>
> >>> On 01/18/2014 08:26 PM, Martin Jansa wrote:
> >>>> On Thu, Jan 16, 2014 at 05:33:20PM +0800, Robert Yang wrote:
> >>>>>
> >>>>>
> >>>>> On 01/16/2014 05:25 PM, Martin Jansa wrote:
> >>>>>> On Thu, Jan 16, 2014 at 10:14:30AM +0800, Robert Yang wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> On 01/16/2014 12:55 AM, Martin Jansa wrote:
> >>>>>>>> On Tue, Jan 14, 2014 at 12:20:14AM +0800, Robert Yang wrote:
> >>>>>>>>> * The archive*.bbclass didn't work, and there were a few problems, for
> >>>>>>>>>       example:
> >>>>>>>>>       1) There were a few duplicated code
> >>>>>>>>>       2) There was no src_dir.org (or orig), but the diff command still
> >>>>>>>>> use
> >>>>>>>>>          it, and it is not easy to fix this issue if we don't change a lot
> >>>>>>>>>          of the code.
> >>>>>>>>>       3) It didn't archive the source for the native or gcc
> >>>>>>>>>       4) The work flow is not very well
> >>>>>>>>>       5) The "subprocess.call('fakeroot cp xxxx'" should be removed
> >>>>>>>>>       6) And others ...
> >>>>>>>>>
> >>>>>>>>> * So that we have to refactor it, the benefits are:
> >>>>>>>>>       1) Fix the problems and make it work well.
> >>>>>>>>>       2) Reduce more than 400 lines in total.
> >>>>>>>>>       3) Make it easy to use.
> >>>>>>>>>
> >>>>>>>>> // Robert
> >>>>>>>>>
> >>>>>>>>> The following changes since commit
> >>>>>>>>> 1b636173ca88e5ccca1992f9a12367a1189fa674:
> >>>>>>>>>
> >>>>>>>>>       bitbake: toaster: Toaster GUI, generic search, filter and order
> >>>>>>>>> (2014-01-10 15:20:26 +0000)
> >>>>>>>>>
> >>>>>>>>> are available in the git repository at:
> >>>>>>>>>
> >>>>>>>>>       git://git.pokylinux.org/poky-contrib rbt/archiver
> >>>>>>>>>       http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
> >>>>>>>>
> >>>>>>>> Can you send pull-request for oe-core?
> >>>>>>>>
> >>>>>>>
> >>>>>>> Here it is:
> >>>>>>> git://git.pokylinux.org/poky-contrib rbt/ar_oe
> >>>>>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rbt/ar_oe
> >>>>>>
> >>>>>> This still looks like poky-contrib (so it will download whole
> >>>>>> poky-contrib when I add it as remote to oe-core repo).
> >>>>>>
> >>>>>
> >>>>> Sorry, I can't create repo on oe-contrib, seems that I lost my key.
> >>>>
> >>>> That's sad, you should send new one to khem to replace it, before
> >>>> someone else finds your old key.
> >>>>
> >>>>> How about:
> >>>>>
> >>>>> git fetch git://git.pokylinux.org/poky-contrib rbt/ar_oe
> >>>>
> >>>> That still fetches whole poky repo I'm not interested in.
> >>>>
> >>>> So I've downloaded just archiver.bbclass manually and tested it on my
> >>>> use-case, I can confirm that it doesn't remove required file anymore,
> >>>> but it seems that there is something terribly broken when used in
> >>>> combination with rm_work or multimachine builds.
> >>>>
> >>>> I haven't debugged it yet, but 2nd build (in the same directory) with
> >>>> this new archiver included almost every recipe failed in do_patch task.
> >>>>
> >>>>>>> // Robert
> >>>>>>>
> >>>>>>>>> Robert Yang (5):
> >>>>>>>>>       classes/archive*.bbclass: remove archive-*-source.bbclass
> >>>>>>>>>       archiver.bbclass: refactor it
> >>>>>>>>>       package_rpm.bbclass: archive the source to srpm package
> >>>>>>>>>       archiver.bbclass: move a few code to copyleft_compliance.bbclass
> >>>>>>>>>       local.conf.sample.extended: update for the archiver
> >>>>>>>>>
> >>>>>>>>>      meta-yocto/conf/local.conf.sample.extended     |  72 +--
> >>>>>>>>>      meta/classes/archive-configured-source.bbclass |  65 ---
> >>>>>>>>>      meta/classes/archive-original-source.bbclass   |  65 ---
> >>>>>>>>>      meta/classes/archive-patched-source.bbclass    |  65 ---
> >>>>>>>>>      meta/classes/archiver.bbclass                  | 723
> >>>>>>>>> ++++++++-----------------
> >>>>>>>>>      meta/classes/copyleft_compliance.bbclass       |  55 +-
> >>>>>>>>>      meta/classes/package_rpm.bbclass               |  31 +-
> >>>>>>>>>      7 files changed, 316 insertions(+), 760 deletions(-)
> >>>>>>>>>      delete mode 100644 meta/classes/archive-configured-source.bbclass
> >>>>>>>>>      delete mode 100644 meta/classes/archive-original-source.bbclass
> >>>>>>>>>      delete mode 100644 meta/classes/archive-patched-source.bbclass
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> 1.8.3.1
> >>>>>>>>>
> >>>>>>>>> _______________________________________________
> >>>>>>>>> Openembedded-core mailing list
> >>>>>>>>> Openembedded-core@lists.openembedded.org
> >>>>>>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >
> >
Robert Yang - Feb. 13, 2014, 9:54 a.m.
Please hold on this, the recent changes of the prefuncs and postfuncs
on bitbake makes it doesn't work on gcc-*, I will send a V2.

// Robert

On 01/14/2014 12:20 AM, Robert Yang wrote:
> * The archive*.bbclass didn't work, and there were a few problems, for
>    example:
>    1) There were a few duplicated code
>    2) There was no src_dir.org (or orig), but the diff command still use
>       it, and it is not easy to fix this issue if we don't change a lot
>       of the code.
>    3) It didn't archive the source for the native or gcc
>    4) The work flow is not very well
>    5) The "subprocess.call('fakeroot cp xxxx'" should be removed
>    6) And others ...
>
> * So that we have to refactor it, the benefits are:
>    1) Fix the problems and make it work well.
>    2) Reduce more than 400 lines in total.
>    3) Make it easy to use.
>
> // Robert
>
> The following changes since commit 1b636173ca88e5ccca1992f9a12367a1189fa674:
>
>    bitbake: toaster: Toaster GUI, generic search, filter and order (2014-01-10 15:20:26 +0000)
>
> are available in the git repository at:
>
>    git://git.pokylinux.org/poky-contrib rbt/archiver
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/archiver
>
> Robert Yang (5):
>    classes/archive*.bbclass: remove archive-*-source.bbclass
>    archiver.bbclass: refactor it
>    package_rpm.bbclass: archive the source to srpm package
>    archiver.bbclass: move a few code to copyleft_compliance.bbclass
>    local.conf.sample.extended: update for the archiver
>
>   meta-yocto/conf/local.conf.sample.extended     |  72 +--
>   meta/classes/archive-configured-source.bbclass |  65 ---
>   meta/classes/archive-original-source.bbclass   |  65 ---
>   meta/classes/archive-patched-source.bbclass    |  65 ---
>   meta/classes/archiver.bbclass                  | 723 ++++++++-----------------
>   meta/classes/copyleft_compliance.bbclass       |  55 +-
>   meta/classes/package_rpm.bbclass               |  31 +-
>   7 files changed, 316 insertions(+), 760 deletions(-)
>   delete mode 100644 meta/classes/archive-configured-source.bbclass
>   delete mode 100644 meta/classes/archive-original-source.bbclass
>   delete mode 100644 meta/classes/archive-patched-source.bbclass
>
Henning Heinold - Feb. 13, 2014, 10:14 a.m.
On Thu, Feb 13, 2014 at 05:54:50PM +0800, Robert Yang wrote:
> 
> Please hold on this, the recent changes of the prefuncs and postfuncs
> on bitbake makes it doesn't work on gcc-*, I will send a V2.
> 
> // Robert
> 

Hi Robert,

do not forget the sstate problems which of course happens on dora branch too.

Bye Henning