Patchwork [0/5] network based PR service(revised)

login
register
mail settings
Submitter Lianhao Lu
Date May 26, 2011, 11:55 a.m.
Message ID <cover.1306401007.git.lianhao.lu@intel.com>
Download mbox
Permalink /patch/4815/
State New, archived
Headers show

Pull-request

git://git.pokylinux.org/poky-contrib llu/PR-service

Comments

Lianhao Lu - May 26, 2011, 11:55 a.m.
This is a modification of the original patch to enable network based PR service. 

The main difference between this series of patch to the original one is that 
this one uses the PKGR/PKGV during the package feed creation. It also made the 
PR service disabled by default(by leaving the PRSERV_HOST and PRSERV_PORT 
commented out) and revised some function names.

The following features from the previous patch comments are not included in this 
series:
1) automatically launch PR service in localhost use case.
2) respect OVERIDE to allow setting PRSERV_HOST/PORT per recipe basis
3) write permission control in the PR service.
4) way to sync up local PR database witht the central server.

I'll continue to work on some of those features. But I'd like to send this patch 
now to meet the schedule time window, and anyone interested can experiment and 
provide some comments.

Below is the excerpt from my last series of patch.
---------------BEGIN of last patch message------------------------------------
This series of 5 patches implemented the network based PR service and enabled 
the poky to use it during the task do_package and do_package_write_xxx. By 
using the network based PR service and the basichash for BB_SIGNATURE_HANDLER, 
the poky user may not need to bump the PR manually everytime he/she changes 
the recipe. The package will get automatically rebuilt and new revision number 
reflecting the change will be included in the package feed.

The first patch "[PATCH 1/5] Added the PR service." implemented the network 
based PR service on the server side. It is a XMLRPC server with the sqlite3 
as the backend database. The users query an automated value from this service 
by giving  a tuple of (version, checksum). The returned value will 
be used as a part of the package revision value during the package feed 
creation process. The following algorihtm is used so this PR service could give 
multiple build servers new values for the new index tuple and same values for 
the same index tuple:

IF the index tuple(version, checksum) is found,  THEN
    return the value.
ELSE 
    find the max value in the database with the same version. 
    increment it and save it into the database along with the index tuple.
    return the incremented value. 
    (Note: If no matching version is found, save and return the value of 0).
ENDIF
 
To start the network based PR service daemon, run the following command after 
"sourcing" the environment file oe-init-build-env:

  bitbake-prserv --start

See bitbake-prserv --help to see a detailed description of the options.

The remaining 4 patches enable the poky to use the PR service. In order to use 
it, the user needs to set 2 varialbes of PRSERV_HOST and PRSERV_PORT (which 
specify the PR service ip address and port, default is "localhost" and "8585" 
respectively) in the file conf/local.conf under the building directory. 
Leaving these 2 variables commented out would disable the poky to use the PR 
service, so the poky would behave exactly like what is now.
------------------END of last patch message----------------------------------

The following changes since commit 2458da25a83df9dc8d849ccf3de7f80fb194a33a:
  Darren Hart (1):
        README.hardware: update installation instructions for beagleboard

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib llu/PR-service
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/PR-service

Lianhao Lu (5):
  Added the PR service.
  conf/bitbake.conf: Added variables for PR service.
  classes/package(prserv).bbclass: Added PR service support.
  classes/package_xxx.class: Added PR service support.
  meta-yocto/local.conf.sample: Added PRSERV_HOST and PRSERV_PORT.

 bitbake/bin/bitbake-prserv        |   53 ++++++++++
 bitbake/lib/prserv/__init__.py    |   11 ++
 bitbake/lib/prserv/db.py          |  100 +++++++++++++++++++
 bitbake/lib/prserv/serv.py        |  198 +++++++++++++++++++++++++++++++++++++
 meta-yocto/conf/local.conf.sample |    5 +
 meta/classes/package.bbclass      |   51 +++++++---
 meta/classes/package_deb.bbclass  |    6 +-
 meta/classes/package_ipk.bbclass  |    6 +-
 meta/classes/package_rpm.bbclass  |   10 +-
 meta/classes/package_tar.bbclass  |    4 +-
 meta/classes/prserv.bbclass       |   29 ++++++
 meta/conf/bitbake.conf            |   18 +++-
 12 files changed, 461 insertions(+), 30 deletions(-)
 create mode 100755 bitbake/bin/bitbake-prserv
 create mode 100644 bitbake/lib/prserv/__init__.py
 create mode 100644 bitbake/lib/prserv/db.py
 create mode 100644 bitbake/lib/prserv/serv.py
 create mode 100644 meta/classes/prserv.bbclass
Koen Kooi - May 26, 2011, 12:07 p.m.
Op 26 mei 2011, om 13:55 heeft Lianhao Lu het volgende geschreven:

> This is a modification of the original patch to enable network based PR service. 
> 
> The main difference between this series of patch to the original one is that 
> this one uses the PKGR/PKGV during the package feed creation. It also made the 
> PR service disabled by default(by leaving the PRSERV_HOST and PRSERV_PORT 
> commented out) and revised some function names.
> 
> The following features from the previous patch comments are not included in this 
> series:
> 1) automatically launch PR service in localhost use case.
> 2) respect OVERIDE to allow setting PRSERV_HOST/PORT per recipe basis
> 3) write permission control in the PR service.
> 4) way to sync up local PR database witht the central server.

Does it support going thru HTTP_PROXY like the bitbake fetchers? It's not a must-have, but it would be nice if I can have the internal builds consistent with angstrom upstream.

regards,

Koen
Lianhao Lu - May 26, 2011, 12:29 p.m.
Koen Kooi wrote on 2011-05-26:
> 
> Op 26 mei 2011, om 13:55 heeft Lianhao Lu het volgende geschreven:
> 
>> This is a modification of the original patch to enable network based PR
>> service.
>> 
>> The main difference between this series of patch to the original one is
>> that this one uses the PKGR/PKGV during the package feed creation. It
>> also made the PR service disabled by default(by leaving the PRSERV_HOST
>> and PRSERV_PORT commented out) and revised some function names.
>> 
>> The following features from the previous patch comments are not
>> included in this
>> series:
>> 1) automatically launch PR service in localhost use case.
>> 2) respect OVERIDE to allow setting PRSERV_HOST/PORT per recipe
>> basis
>> 3) write permission control in the PR service.
>> 4) way to sync up local PR database witht the central server.
> 
> Does it support going thru HTTP_PROXY like the bitbake fetchers? It's
> not a must-have, but it would be nice if I can have the internal
> builds consistent with angstrom upstream.
> 

It doesn't support now. But I think we can have this by implementing a custom transport, by leverage the urllib2.

Best Regards,
Lianhao