Patchwork [00/11] Add PR service and integrate PKGV/PKGR from OE

login
mail settings
Submitter Lianhao Lu
Date May 27, 2011, 6:31 a.m.
Message ID <cover.1306477652.git.lianhao.lu@intel.com>
Download mbox
Permalink /patch/4873/
State New, archived
Headers show

Pull-request

git://git.pokylinux.org/poky-contrib llu/PR-service

Comments

Lianhao Lu - May 27, 2011, 6:31 a.m.
This is a modification of the original patch to enable network based PR service. 

The main difference between this series of patch to the original one is that 
this one uses the PKGR/PKGV during the package feed creation, and use EXTENDPKGV 
instead of EXTENDPV for package dependencies.

It also made the PR service disabled by default(by leaving the PRSERV_HOST and 
PRSERV_PORT commented out) and revised some function names.

The following features from the previous patch comments are not included in this 
series:
1) automatically launch PR service in localhost use case.
2) respect OVERIDE to allow setting PRSERV_HOST/PORT per recipe basis
3) write permission control in the PR service.
4) way to sync up local PR database witht the central server.
5) added proxy support for PR service.

I'll continue to work on some of those features. But I'd like to send this patch 
now to meet the schedule time window, and anyone interested can experiment and 
provide some comments.

Below is the excerpt from my last series of patch.
---------------BEGIN of last patch message------------------------------------
This series of 5 patches implemented the network based PR service and enabled 
the poky to use it during the task do_package and do_package_write_xxx. By 
using the network based PR service and the basichash for BB_SIGNATURE_HANDLER, 
the poky user may not need to bump the PR manually everytime he/she changes 
the recipe. The package will get automatically rebuilt and new revision number 
reflecting the change will be included in the package feed.

The first patch "[PATCH 1/5] Added the PR service." implemented the network 
based PR service on the server side. It is a XMLRPC server with the sqlite3 
as the backend database. The users query an automated value from this service 
by giving  a tuple of (version, checksum). The returned value will 
be used as a part of the package revision value during the package feed 
creation process. The following algorihtm is used so this PR service could give 
multiple build servers new values for the new index tuple and same values for 
the same index tuple:

IF the index tuple(version, checksum) is found,  THEN
    return the value.
ELSE 
    find the max value in the database with the same version. 
    increment it and save it into the database along with the index tuple.
    return the incremented value. 
    (Note: If no matching version is found, save and return the value of 0).
ENDIF
 
To start the network based PR service daemon, run the following command after 
"sourcing" the environment file oe-init-build-env:

  bitbake-prserv --start

See bitbake-prserv --help to see a detailed description of the options.

The remaining 4 patches enable the poky to use the PR service. In order to use 
it, the user needs to set 2 varialbes of PRSERV_HOST and PRSERV_PORT (which 
specify the PR service ip address and port, default is "localhost" and "8585" 
respectively) in the file conf/local.conf under the building directory. 
Leaving these 2 variables commented out would disable the poky to use the PR 
service, so the poky would behave exactly like what is now.
------------------END of last patch message----------------------------------

The following changes since commit 9bccbc5fbc71b331911b558a2ba15e0a6d0046b4:
  Nitin A Kamble (1):
        tcmode-default: workaround for gcc 4.6.0 failure on beagleboard

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib llu/PR-service
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service

Lianhao Lu (11):
  Added the PR service.
  conf/bitbake.conf: Added variables for PR service.
  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
  classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
  classes/package_xxx.class: Use PKGE/PKGV/PKGR.
  udev: use EXTENDPKGV instead of EXTENDPV.
  xcb: use EXTENDPKGV instead of EXTENDPV.
  xorg-proto: use EXTENDPKGV instead of EXTENDPV.
  xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
  linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
  conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.

 bitbake/bin/bitbake-prserv                         |   53 ++++++
 bitbake/lib/prserv/__init__.py                     |   11 +
 bitbake/lib/prserv/db.py                           |  100 ++++++++++
 bitbake/lib/prserv/serv.py                         |  198 ++++++++++++++++++++
 meta-yocto/conf/local.conf.sample                  |    5 +
 meta/classes/package.bbclass                       |   51 ++++--
 meta/classes/package_deb.bbclass                   |    8 +-
 meta/classes/package_ipk.bbclass                   |    8 +-
 meta/classes/package_rpm.bbclass                   |   14 +-
 meta/classes/package_tar.bbclass                   |    4 +-
 meta/classes/prserv.bbclass                        |   29 +++
 meta/conf/bitbake.conf                             |   23 ++-
 meta/recipes-core/udev/udev-new.inc                |    2 +-
 meta/recipes-graphics/xcb/libpthread-stubs_0.3.bb  |    2 +-
 meta/recipes-graphics/xcb/xcb-proto.inc            |    2 +-
 .../xorg-proto/xorg-proto-common.inc               |    2 +-
 .../xorg-util/util-macros_1.13.0.bb                |    2 +-
 .../linux-libc-headers/linux-libc-headers.inc      |    2 +-
 18 files changed, 473 insertions(+), 43 deletions(-)
 create mode 100755 bitbake/bin/bitbake-prserv
 create mode 100644 bitbake/lib/prserv/__init__.py
 create mode 100644 bitbake/lib/prserv/db.py
 create mode 100644 bitbake/lib/prserv/serv.py
 create mode 100644 meta/classes/prserv.bbclass
Richard Purdie - May 27, 2011, 5 p.m.
Hi Lianhao,

On Fri, 2011-05-27 at 14:31 +0800, Lianhao Lu wrote:
>   git://git.pokylinux.org/poky-contrib llu/PR-service
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service
> 
> Lianhao Lu (11):
>   Added the PR service.
>   conf/bitbake.conf: Added variables for PR service.
>   meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
>   classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
>   classes/package_xxx.class: Use PKGE/PKGV/PKGR.
>   udev: use EXTENDPKGV instead of EXTENDPV.
>   xcb: use EXTENDPKGV instead of EXTENDPV.
>   xorg-proto: use EXTENDPKGV instead of EXTENDPV.
>   xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
>   linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
>   conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.

I merged this but I did make some tweaks to both the patch series and
one of the patches. The changes I made were:

a) Squash the EXTENDPV and EXTENDPEVER changes into the base patch
making all the associated changes in one go.

b) Not include the DISTRO_PR variable for now

c) Not change the default signature handler to basichash.

For c), I'd suggest a follow up patch changing the default in poky.conf
in meta-yocto and we'll test this there. This should wait until the
PACKAGE_ARCH = "all" sstate issues we've been seeing are addressed
though.

Cheers,

Richard
Koen Kooi - May 27, 2011, 6:48 p.m.
Op 27 mei 2011, om 19:00 heeft Richard Purdie het volgende geschreven:

> Hi Lianhao,
> 
> On Fri, 2011-05-27 at 14:31 +0800, Lianhao Lu wrote:
>>  git://git.pokylinux.org/poky-contrib llu/PR-service
>>  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service
>> 
>> Lianhao Lu (11):
>>  Added the PR service.
>>  conf/bitbake.conf: Added variables for PR service.
>>  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
>>  classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
>>  classes/package_xxx.class: Use PKGE/PKGV/PKGR.
>>  udev: use EXTENDPKGV instead of EXTENDPV.
>>  xcb: use EXTENDPKGV instead of EXTENDPV.
>>  xorg-proto: use EXTENDPKGV instead of EXTENDPV.
>>  xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
>>  linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
>>  conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.
> 
> I merged this but I did make some tweaks to both the patch series and
> one of the patches. The changes I made were:
> 
> a) Squash the EXTENDPV and EXTENDPEVER changes into the base patch
> making all the associated changes in one go.
> 
> b) Not include the DISTRO_PR variable for now
> 
> c) Not change the default signature handler to basichash.
> 
> For c), I'd suggest a follow up patch changing the default in poky.conf
> in meta-yocto and we'll test this there. This should wait until the
> PACKAGE_ARCH = "all" sstate issues we've been seeing are addressed
> though.

On Fedora 14 I get this when trying to rebuild console-image:

ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
ERROR: Function 'package_get_auto_pr' failed
ERROR: Logfile of failure stored in: /home/koen/angstrom-core/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/dbus-1.4.1-r5/temp/log.do_package.24742
Log data follows:
| ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-core/dbus/dbus_1.4.1.bb:
| ImportError: No module named xmlrpc
| 
| ERROR: The stack trace of python calls that resulted in this exception/failure was:
| ERROR:   File "package_get_auto_pr", line 11, in <module>
| ERROR:
| ERROR:   File "package_get_auto_pr", line 4, in package_get_auto_pr
| ERROR:
| ERROR:   File "prserv.bbclass", line 8, in prserv_get_pr_auto
| ERROR:
| ERROR:   File "prserv.bbclass", line 2, in prserv_make_conn
| ERROR:
| ERROR:   File "/home/koen/angstrom-core/sources/bitbake/lib/prserv/serv.py", line 6, in <module>
| ERROR:     import bb.server.xmlrpc
| ERROR:
| ERROR: The code that was being executed was:
| ERROR:      0007:                     return
| ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
| ERROR:      0009:
| ERROR:      0010:
| ERROR:  *** 0011:package_get_auto_pr(d)
| ERROR:      0012:
| ERROR: (file: 'package_get_auto_pr', lineno: 11, function: <module>)
| ERROR:      0001:
| ERROR:      0002:def package_get_auto_pr(d):
| ERROR:      0003:     if d.getVar('USE_PR_SERV', True):
| ERROR:  *** 0004:             auto_pr=prserv_get_pr_auto(d)
| ERROR:      0005:             if auto_pr is None:
| ERROR:      0006:                     bb.fatal("Can NOT get auto PR revision from remote PR service")
| ERROR:      0007:                     return
| ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
| ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
| ERROR: Function 'package_get_auto_pr' failed

It doesn't seem to trigger on my debian box, though
Otavio Salvador - May 27, 2011, 7:08 p.m.
Same here.

On Fri, May 27, 2011 at 18:48, Koen Kooi <koen@dominion.thruhere.net> wrote:
>
> Op 27 mei 2011, om 19:00 heeft Richard Purdie het volgende geschreven:
>
>> Hi Lianhao,
>>
>> On Fri, 2011-05-27 at 14:31 +0800, Lianhao Lu wrote:
>>>  git://git.pokylinux.org/poky-contrib llu/PR-service
>>>  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service
>>>
>>> Lianhao Lu (11):
>>>  Added the PR service.
>>>  conf/bitbake.conf: Added variables for PR service.
>>>  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
>>>  classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
>>>  classes/package_xxx.class: Use PKGE/PKGV/PKGR.
>>>  udev: use EXTENDPKGV instead of EXTENDPV.
>>>  xcb: use EXTENDPKGV instead of EXTENDPV.
>>>  xorg-proto: use EXTENDPKGV instead of EXTENDPV.
>>>  xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
>>>  linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
>>>  conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.
>>
>> I merged this but I did make some tweaks to both the patch series and
>> one of the patches. The changes I made were:
>>
>> a) Squash the EXTENDPV and EXTENDPEVER changes into the base patch
>> making all the associated changes in one go.
>>
>> b) Not include the DISTRO_PR variable for now
>>
>> c) Not change the default signature handler to basichash.
>>
>> For c), I'd suggest a follow up patch changing the default in poky.conf
>> in meta-yocto and we'll test this there. This should wait until the
>> PACKAGE_ARCH = "all" sstate issues we've been seeing are addressed
>> though.
>
> On Fedora 14 I get this when trying to rebuild console-image:
>
> ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
> ERROR: Function 'package_get_auto_pr' failed
> ERROR: Logfile of failure stored in: /home/koen/angstrom-core/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/dbus-1.4.1-r5/temp/log.do_package.24742
> Log data follows:
> | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-core/dbus/dbus_1.4.1.bb:
> | ImportError: No module named xmlrpc
> |
> | ERROR: The stack trace of python calls that resulted in this exception/failure was:
> | ERROR:   File "package_get_auto_pr", line 11, in <module>
> | ERROR:
> | ERROR:   File "package_get_auto_pr", line 4, in package_get_auto_pr
> | ERROR:
> | ERROR:   File "prserv.bbclass", line 8, in prserv_get_pr_auto
> | ERROR:
> | ERROR:   File "prserv.bbclass", line 2, in prserv_make_conn
> | ERROR:
> | ERROR:   File "/home/koen/angstrom-core/sources/bitbake/lib/prserv/serv.py", line 6, in <module>
> | ERROR:     import bb.server.xmlrpc
> | ERROR:
> | ERROR: The code that was being executed was:
> | ERROR:      0007:                     return
> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
> | ERROR:      0009:
> | ERROR:      0010:
> | ERROR:  *** 0011:package_get_auto_pr(d)
> | ERROR:      0012:
> | ERROR: (file: 'package_get_auto_pr', lineno: 11, function: <module>)
> | ERROR:      0001:
> | ERROR:      0002:def package_get_auto_pr(d):
> | ERROR:      0003:     if d.getVar('USE_PR_SERV', True):
> | ERROR:  *** 0004:             auto_pr=prserv_get_pr_auto(d)
> | ERROR:      0005:             if auto_pr is None:
> | ERROR:      0006:                     bb.fatal("Can NOT get auto PR revision from remote PR service")
> | ERROR:      0007:                     return
> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
> | ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
> | ERROR: Function 'package_get_auto_pr' failed
>
> It doesn't seem to trigger on my debian box, though
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Koen Kooi - May 27, 2011, 7:13 p.m.
FWIW, F14 has python 2.7


Op 27 mei 2011, om 21:08 heeft Otavio Salvador het volgende geschreven:

> Same here.
> 
> On Fri, May 27, 2011 at 18:48, Koen Kooi <koen@dominion.thruhere.net> wrote:
>> 
>> Op 27 mei 2011, om 19:00 heeft Richard Purdie het volgende geschreven:
>> 
>>> Hi Lianhao,
>>> 
>>> On Fri, 2011-05-27 at 14:31 +0800, Lianhao Lu wrote:
>>>>  git://git.pokylinux.org/poky-contrib llu/PR-service
>>>>  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service
>>>> 
>>>> Lianhao Lu (11):
>>>>  Added the PR service.
>>>>  conf/bitbake.conf: Added variables for PR service.
>>>>  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
>>>>  classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
>>>>  classes/package_xxx.class: Use PKGE/PKGV/PKGR.
>>>>  udev: use EXTENDPKGV instead of EXTENDPV.
>>>>  xcb: use EXTENDPKGV instead of EXTENDPV.
>>>>  xorg-proto: use EXTENDPKGV instead of EXTENDPV.
>>>>  xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
>>>>  linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
>>>>  conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.
>>> 
>>> I merged this but I did make some tweaks to both the patch series and
>>> one of the patches. The changes I made were:
>>> 
>>> a) Squash the EXTENDPV and EXTENDPEVER changes into the base patch
>>> making all the associated changes in one go.
>>> 
>>> b) Not include the DISTRO_PR variable for now
>>> 
>>> c) Not change the default signature handler to basichash.
>>> 
>>> For c), I'd suggest a follow up patch changing the default in poky.conf
>>> in meta-yocto and we'll test this there. This should wait until the
>>> PACKAGE_ARCH = "all" sstate issues we've been seeing are addressed
>>> though.
>> 
>> On Fedora 14 I get this when trying to rebuild console-image:
>> 
>> ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
>> ERROR: Function 'package_get_auto_pr' failed
>> ERROR: Logfile of failure stored in: /home/koen/angstrom-core/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/dbus-1.4.1-r5/temp/log.do_package.24742
>> Log data follows:
>> | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-core/dbus/dbus_1.4.1.bb:
>> | ImportError: No module named xmlrpc
>> |
>> | ERROR: The stack trace of python calls that resulted in this exception/failure was:
>> | ERROR:   File "package_get_auto_pr", line 11, in <module>
>> | ERROR:
>> | ERROR:   File "package_get_auto_pr", line 4, in package_get_auto_pr
>> | ERROR:
>> | ERROR:   File "prserv.bbclass", line 8, in prserv_get_pr_auto
>> | ERROR:
>> | ERROR:   File "prserv.bbclass", line 2, in prserv_make_conn
>> | ERROR:
>> | ERROR:   File "/home/koen/angstrom-core/sources/bitbake/lib/prserv/serv.py", line 6, in <module>
>> | ERROR:     import bb.server.xmlrpc
>> | ERROR:
>> | ERROR: The code that was being executed was:
>> | ERROR:      0007:                     return
>> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
>> | ERROR:      0009:
>> | ERROR:      0010:
>> | ERROR:  *** 0011:package_get_auto_pr(d)
>> | ERROR:      0012:
>> | ERROR: (file: 'package_get_auto_pr', lineno: 11, function: <module>)
>> | ERROR:      0001:
>> | ERROR:      0002:def package_get_auto_pr(d):
>> | ERROR:      0003:     if d.getVar('USE_PR_SERV', True):
>> | ERROR:  *** 0004:             auto_pr=prserv_get_pr_auto(d)
>> | ERROR:      0005:             if auto_pr is None:
>> | ERROR:      0006:                     bb.fatal("Can NOT get auto PR revision from remote PR service")
>> | ERROR:      0007:                     return
>> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
>> | ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
>> | ERROR: Function 'package_get_auto_pr' failed
>> 
>> It doesn't seem to trigger on my debian box, though
>> 
>> 
>> 
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>> 
> 
> 
> 
> -- 
> Otavio Salvador                             O.S. Systems
> E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
> Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Joshua Lock - May 27, 2011, 8:09 p.m.
On Fri, 2011-05-27 at 20:48 +0200, Koen Kooi wrote:
> Op 27 mei 2011, om 19:00 heeft Richard Purdie het volgende geschreven:
> 
> > Hi Lianhao,
> > 
> > On Fri, 2011-05-27 at 14:31 +0800, Lianhao Lu wrote:
> >>  git://git.pokylinux.org/poky-contrib llu/PR-service
> >>  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service
> >> 
> >> Lianhao Lu (11):
> >>  Added the PR service.
> >>  conf/bitbake.conf: Added variables for PR service.
> >>  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
> >>  classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
> >>  classes/package_xxx.class: Use PKGE/PKGV/PKGR.
> >>  udev: use EXTENDPKGV instead of EXTENDPV.
> >>  xcb: use EXTENDPKGV instead of EXTENDPV.
> >>  xorg-proto: use EXTENDPKGV instead of EXTENDPV.
> >>  xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
> >>  linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
> >>  conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.
> > 
> > I merged this but I did make some tweaks to both the patch series and
> > one of the patches. The changes I made were:
> > 
> > a) Squash the EXTENDPV and EXTENDPEVER changes into the base patch
> > making all the associated changes in one go.
> > 
> > b) Not include the DISTRO_PR variable for now
> > 
> > c) Not change the default signature handler to basichash.
> > 
> > For c), I'd suggest a follow up patch changing the default in poky.conf
> > in meta-yocto and we'll test this there. This should wait until the
> > PACKAGE_ARCH = "all" sstate issues we've been seeing are addressed
> > though.
> 
> On Fedora 14 I get this when trying to rebuild console-image:
> 
> ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
> ERROR: Function 'package_get_auto_pr' failed
> ERROR: Logfile of failure stored in: /home/koen/angstrom-core/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/dbus-1.4.1-r5/temp/log.do_package.24742
> Log data follows:
> | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-core/dbus/dbus_1.4.1.bb:
> | ImportError: No module named xmlrpc
> | 
> | ERROR: The stack trace of python calls that resulted in this exception/failure was:
> | ERROR:   File "package_get_auto_pr", line 11, in <module>
> | ERROR:
> | ERROR:   File "package_get_auto_pr", line 4, in package_get_auto_pr
> | ERROR:
> | ERROR:   File "prserv.bbclass", line 8, in prserv_get_pr_auto
> | ERROR:
> | ERROR:   File "prserv.bbclass", line 2, in prserv_make_conn
> | ERROR:
> | ERROR:   File "/home/koen/angstrom-core/sources/bitbake/lib/prserv/serv.py", line 6, in <module>
> | ERROR:     import bb.server.xmlrpc

Upstream BitBake still has the xmlrpc server removed. The version in
Poky's BitBake is functional (I use it daily).

Attached are two patches against BitBake master to add back the xmlrpc
server (copied from Poky's tree) and fix the uievent.

> | ERROR:
> | ERROR: The code that was being executed was:
> | ERROR:      0007:                     return
> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
> | ERROR:      0009:
> | ERROR:      0010:
> | ERROR:  *** 0011:package_get_auto_pr(d)
> | ERROR:      0012:
> | ERROR: (file: 'package_get_auto_pr', lineno: 11, function: <module>)
> | ERROR:      0001:
> | ERROR:      0002:def package_get_auto_pr(d):
> | ERROR:      0003:     if d.getVar('USE_PR_SERV', True):
> | ERROR:  *** 0004:             auto_pr=prserv_get_pr_auto(d)
> | ERROR:      0005:             if auto_pr is None:
> | ERROR:      0006:                     bb.fatal("Can NOT get auto PR revision from remote PR service")
> | ERROR:      0007:                     return
> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
> | ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
> | ERROR: Function 'package_get_auto_pr' failed
> 
> It doesn't seem to trigger on my debian box, though

Are you using Poky on the Debian box?

Cheers,
Joshua
Khem Raj - May 27, 2011, 8:24 p.m.
On Fri, May 27, 2011 at 1:09 PM, Joshua Lock <josh@linux.intel.com> wrote:
> On Fri, 2011-05-27 at 20:48 +0200, Koen Kooi wrote:
>> Op 27 mei 2011, om 19:00 heeft Richard Purdie het volgende geschreven:
>>
>> > Hi Lianhao,
>> >
>> > On Fri, 2011-05-27 at 14:31 +0800, Lianhao Lu wrote:
>> >>  git://git.pokylinux.org/poky-contrib llu/PR-service
>> >>  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service
>> >>
>> >> Lianhao Lu (11):
>> >>  Added the PR service.
>> >>  conf/bitbake.conf: Added variables for PR service.
>> >>  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.
>> >>  classes/package(prserv).bbclass: Get PRAUTO and use PKGV/PKGR.
>> >>  classes/package_xxx.class: Use PKGE/PKGV/PKGR.
>> >>  udev: use EXTENDPKGV instead of EXTENDPV.
>> >>  xcb: use EXTENDPKGV instead of EXTENDPV.
>> >>  xorg-proto: use EXTENDPKGV instead of EXTENDPV.
>> >>  xorg-util/util-macros: use EXTENDPKGV instead of EXTENDPV.
>> >>  linux-libc-headers: use EXTENDPKGV instead of EXTENDPV.
>> >>  conf/bitbake.conf: Removed EXTENDPV and EXTENDPEVER.
>> >
>> > I merged this but I did make some tweaks to both the patch series and
>> > one of the patches. The changes I made were:
>> >
>> > a) Squash the EXTENDPV and EXTENDPEVER changes into the base patch
>> > making all the associated changes in one go.
>> >
>> > b) Not include the DISTRO_PR variable for now
>> >
>> > c) Not change the default signature handler to basichash.
>> >
>> > For c), I'd suggest a follow up patch changing the default in poky.conf
>> > in meta-yocto and we'll test this there. This should wait until the
>> > PACKAGE_ARCH = "all" sstate issues we've been seeing are addressed
>> > though.
>>
>> On Fedora 14 I get this when trying to rebuild console-image:
>>
>> ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
>> ERROR: Function 'package_get_auto_pr' failed
>> ERROR: Logfile of failure stored in: /home/koen/angstrom-core/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/dbus-1.4.1-r5/temp/log.do_package.24742
>> Log data follows:
>> | ERROR: Error executing a python function in /home/koen/angstrom-core/sources/openembedded-core/meta/recipes-core/dbus/dbus_1.4.1.bb:
>> | ImportError: No module named xmlrpc
>> |
>> | ERROR: The stack trace of python calls that resulted in this exception/failure was:
>> | ERROR:   File "package_get_auto_pr", line 11, in <module>
>> | ERROR:
>> | ERROR:   File "package_get_auto_pr", line 4, in package_get_auto_pr
>> | ERROR:
>> | ERROR:   File "prserv.bbclass", line 8, in prserv_get_pr_auto
>> | ERROR:
>> | ERROR:   File "prserv.bbclass", line 2, in prserv_make_conn
>> | ERROR:
>> | ERROR:   File "/home/koen/angstrom-core/sources/bitbake/lib/prserv/serv.py", line 6, in <module>
>> | ERROR:     import bb.server.xmlrpc
>
> Upstream BitBake still has the xmlrpc server removed. The version in
> Poky's BitBake is functional (I use it daily).
>

I use bb master and oe-core

> Attached are two patches against BitBake master to add back the xmlrpc
> server (copied from Poky's tree) and fix the uievent.
>
>> | ERROR:
>> | ERROR: The code that was being executed was:
>> | ERROR:      0007:                     return
>> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
>> | ERROR:      0009:
>> | ERROR:      0010:
>> | ERROR:  *** 0011:package_get_auto_pr(d)
>> | ERROR:      0012:
>> | ERROR: (file: 'package_get_auto_pr', lineno: 11, function: <module>)
>> | ERROR:      0001:
>> | ERROR:      0002:def package_get_auto_pr(d):
>> | ERROR:      0003:     if d.getVar('USE_PR_SERV', True):
>> | ERROR:  *** 0004:             auto_pr=prserv_get_pr_auto(d)
>> | ERROR:      0005:             if auto_pr is None:
>> | ERROR:      0006:                     bb.fatal("Can NOT get auto PR revision from remote PR service")
>> | ERROR:      0007:                     return
>> | ERROR:      0008:             d.setVar('PRAUTO',str(auto_pr))
>> | ERROR: (file: 'package_get_auto_pr', lineno: 4, function: package_get_auto_pr)
>> | ERROR: Function 'package_get_auto_pr' failed
>>
>> It doesn't seem to trigger on my debian box, though
>
> Are you using Poky on the Debian box?

ubuntu 64bit oe-core no poky

>
> Cheers,
> Joshua
> --
> Joshua Lock
>        Yocto Project Build Monkey
>        Intel Open Source Technology Centre
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
Joshua Lock - May 27, 2011, 8:54 p.m.
On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
> Attached are two patches against BitBake master to add back the xmlrpc
> server (copied from Poky's tree) and fix the uievent.

Seems like my patches didn't make it through? *tries again*

Cheers,
Joshua
Joshua Lock - May 27, 2011, 9 p.m.
On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
> Upstream BitBake still has the xmlrpc server removed. The version in
> Poky's BitBake is functional (I use it daily).
> 
> Attached are two patches against BitBake master to add back the xmlrpc
> server (copied from Poky's tree) and fix the uievent.

Something in the pipeline is eating patches.

I'll send them as replies to this message.

Joshua
Khem Raj - May 27, 2011, 9:01 p.m.
On May 27, 2011, at 2:00 PM, Joshua Lock <josh@linux.intel.com> wrote:

> On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
>> Upstream BitBake still has the xmlrpc server removed. The version in
>> Poky's BitBake is functional (I use it daily).
>> 
>> Attached are two patches against BitBake master to add back the xmlrpc
>> server (copied from Poky's tree) and fix the uievent.
> 
> Something in the pipeline is eating patches.
> 
> I'll send them as replies to this message.

I think it's the mailing list setup. When I attach a patch it never goes through
Inlining the patch does go through
> 
> Joshua
> -- 
> Joshua Lock
>        Yocto Project Build Monkey
>        Intel Open Source Technology Centre
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Joshua Lock - May 27, 2011, 9:09 p.m.
On Fri, 2011-05-27 at 14:01 -0700, Khem Raj wrote:
> 
> On May 27, 2011, at 2:00 PM, Joshua Lock <josh@linux.intel.com> wrote:
> 
> > On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
> >> Upstream BitBake still has the xmlrpc server removed. The version in
> >> Poky's BitBake is functional (I use it daily).
> >> 
> >> Attached are two patches against BitBake master to add back the xmlrpc
> >> server (copied from Poky's tree) and fix the uievent.
> > 
> > Something in the pipeline is eating patches.
> > 
> > I'll send them as replies to this message.
> 
> I think it's the mailing list setup. When I attach a patch it never goes through
> Inlining the patch does go through
> > 

Yeah. It seems to be the oe-core list mangling patches. The attachments
made it to bitbake-devel fine.

Cheers,
Joshua
Richard Purdie - May 27, 2011, 10:09 p.m.
On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
> Upstream BitBake still has the xmlrpc server removed. The version in
> Poky's BitBake is functional (I use it daily).
> 
> Attached are two patches against BitBake master to add back the xmlrpc
> server (copied from Poky's tree) and fix the uievent.

There have been API changes in upstream which mean this code will fail
to work with bitbake itself although it will work with the pr server.

I've merged the patches as a bandaid to allow things to work but we've
now got two different server backends using different server APIs. I
really just want to highlight I'm aware of this and it will get
addressed next week. Its been on my todo list and is way overdue. The
main priority is to get people's builds with OE-Core working again.

Cheers,

Richard
Khem Raj - May 28, 2011, 1:02 a.m.
On Fri, May 27, 2011 at 3:09 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
>> Upstream BitBake still has the xmlrpc server removed. The version in
>> Poky's BitBake is functional (I use it daily).
>>
>> Attached are two patches against BitBake master to add back the xmlrpc
>> server (copied from Poky's tree) and fix the uievent.
>
> There have been API changes in upstream which mean this code will fail
> to work with bitbake itself although it will work with the pr server.
>
> I've merged the patches as a bandaid to allow things to work but we've
> now got two different server backends using different server APIs. I
> really just want to highlight I'm aware of this and it will get
> addressed next week. Its been on my todo list and is way overdue. The
> main priority is to get people's builds with OE-Core working again.
>

btw. I am at 6c412f009e33e77cbcb5d4881c110e6bdda05282 of bitbake
I still see the problem

> Cheers,
>
> Richard
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Khem Raj - May 28, 2011, 1:05 a.m.
On Fri, May 27, 2011 at 6:02 PM, Khem Raj <raj.khem@gmail.com> wrote:
> On Fri, May 27, 2011 at 3:09 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
>>> Upstream BitBake still has the xmlrpc server removed. The version in
>>> Poky's BitBake is functional (I use it daily).
>>>
>>> Attached are two patches against BitBake master to add back the xmlrpc
>>> server (copied from Poky's tree) and fix the uievent.
>>
>> There have been API changes in upstream which mean this code will fail
>> to work with bitbake itself although it will work with the pr server.
>>
>> I've merged the patches as a bandaid to allow things to work but we've
>> now got two different server backends using different server APIs. I
>> really just want to highlight I'm aware of this and it will get
>> addressed next week. Its been on my todo list and is way overdue. The
>> main priority is to get people's builds with OE-Core working again.
>>
>
> btw. I am at 6c412f009e33e77cbcb5d4881c110e6bdda05282 of bitbake
> I still see the problem
>


here is the error

NOTE: Running task 919 of 925 (ID: 9,
/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/uclibc/uclibc_git.bb,
do_package)
NOTE: package uclibc-0.9.31+0.9.32rc3-r2.2: task do_package: Started
ERROR: Connecting to PR service None:None failed: int() argument must
be a string or a number, not 'NoneType'
ERROR: Function 'package_get_auto_pr' failed
ERROR: Logfile of failure stored in:
/home/kraj/work/angstrom/build/tmp-angstrom_2010_x-uclibc/work/efikamx-angstrom-linux-uclibceabi/uclibc-0.9.31+0.9.32rc3-r2.2/temp/log.do_package.5207
Log data follows:
| ERROR: Connecting to PR service None:None failed: int() argument
must be a string or a number, not 'NoneType'
| ERROR: Function 'package_get_auto_pr' failed
NOTE: package uclibc-0.9.31+0.9.32rc3-r2.2: task do_package: Failed
ERROR: Task 9 (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/uclibc/uclibc_git.bb,
do_package) failed with exit code '1'
ERROR: '/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/uclibc/uclibc_git.bb'
failed


>> Cheers,
>>
>> Richard
>>
>>
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>
Khem Raj - May 28, 2011, 1:34 a.m.
On Fri, May 27, 2011 at 6:05 PM, Khem Raj <raj.khem@gmail.com> wrote:
> On Fri, May 27, 2011 at 6:02 PM, Khem Raj <raj.khem@gmail.com> wrote:
>> On Fri, May 27, 2011 at 3:09 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>>> On Fri, 2011-05-27 at 13:09 -0700, Joshua Lock wrote:
>>>> Upstream BitBake still has the xmlrpc server removed. The version in
>>>> Poky's BitBake is functional (I use it daily).
>>>>
>>>> Attached are two patches against BitBake master to add back the xmlrpc
>>>> server (copied from Poky's tree) and fix the uievent.
>>>
>>> There have been API changes in upstream which mean this code will fail
>>> to work with bitbake itself although it will work with the pr server.
>>>
>>> I've merged the patches as a bandaid to allow things to work but we've
>>> now got two different server backends using different server APIs. I
>>> really just want to highlight I'm aware of this and it will get
>>> addressed next week. Its been on my todo list and is way overdue. The
>>> main priority is to get people's builds with OE-Core working again.
>>>
>>
>> btw. I am at 6c412f009e33e77cbcb5d4881c110e6bdda05282 of bitbake
>> I still see the problem
>>
>
>
> here is the error
>
> NOTE: Running task 919 of 925 (ID: 9,
> /home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/uclibc/uclibc_git.bb,
> do_package)
> NOTE: package uclibc-0.9.31+0.9.32rc3-r2.2: task do_package: Started
> ERROR: Connecting to PR service None:None failed: int() argument must
> be a string or a number, not 'NoneType'
> ERROR: Function 'package_get_auto_pr' failed
> ERROR: Logfile of failure stored in:
> /home/kraj/work/angstrom/build/tmp-angstrom_2010_x-uclibc/work/efikamx-angstrom-linux-uclibceabi/uclibc-0.9.31+0.9.32rc3-r2.2/temp/log.do_package.5207
> Log data follows:
> | ERROR: Connecting to PR service None:None failed: int() argument
> must be a string or a number, not 'NoneType'
> | ERROR: Function 'package_get_auto_pr' failed
> NOTE: package uclibc-0.9.31+0.9.32rc3-r2.2: task do_package: Failed
> ERROR: Task 9 (/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/uclibc/uclibc_git.bb,
> do_package) failed with exit code '1'
> ERROR: '/home/kraj/work/angstrom/sources/openembedded-core/meta/recipes-core/uclibc/uclibc_git.bb'
> failed
>
>

this patch seems to fix it.

http://patches.openembedded.org/patch/4941/

>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>>
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>
>>
>