| Submitter | Dexuan Cui |
|---|---|
| Date | May 13, 2011, 9:53 a.m. |
| Message ID | <abd335f0e36fe2a96316b77d5e04e00b8ddb4ea9.1305279161.git.dexuan.cui@intel.com> |
| Download | mbox | patch |
| Permalink | /patch/3939/ |
| State | New, archived |
| Headers | show |
Comments
On 05/13/2011 02:53 AM, Dexuan Cui wrote: > From: Dexuan Cui<dexuan.cui@intel.com> > > [YOCTO #1024] > Currently package-index.bb only supports ipk. This commit addes the support > for rpm and deb, too. > > ------------------------------ > How to generate and use repos: > > 1) run "bitbake package-index" after building some target, > e.g., core-image-sato-sdk; > > 2) export ${DEPLOY_DIR_RPM}, ${DEPLOY_DIR_IPK} and ${DEPLOY_DIR_DEB} by a > webserver on the host, assuming the host IP is 192.168.7.1, at > http://192.168.7.1/rpm > http://192.168.7.1/ipk > http://192.168.7.1/deb > > 3) inside the target, according to the packaging system (rpm, ipk or deb) used > when we generate the target image, we can use different ways to manage > packages: > > 3.1) RPM > run "zypper addrepo http://192.168.7.1/rpm main; zypper refresh" > to retrieve info about the repo; next, we can use "zypper install/remove" > to manage packages. > > 3.2) IPK > add the repo info into opkg config file, i.e., in > /etc/opkg/arch.conf, we can add something like > "src i586 http://192.168.7.1/ipk/i586", and next, we run "opkg update" to > make opkg update the list of available packages. And later, we can use > "opkg install/remove" to manage packages. > > 3.3) DEB > Currently in target, some important config files, like > /etc/apt/sources.list, for deb/apt are missing. So we can't install/remove package > in target at present. > > Signed-off-by: Dexuan Cui<dexuan.cui@intel.com> > --- > meta/classes/package_deb.bbclass | 9 +++++++++ > meta/classes/package_ipk.bbclass | 12 ++++++++++++ > meta/classes/package_rpm.bbclass | 7 +++++++ > meta/recipes-core/meta/package-index.bb | 8 -------- > 4 files changed, 28 insertions(+), 8 deletions(-) > > diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass > index 4faeb4a..4cc0b69 100644 > --- a/meta/classes/package_deb.bbclass > +++ b/meta/classes/package_deb.bbclass > @@ -431,3 +431,12 @@ python do_package_write_deb () { > do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}" > addtask package_write_deb before do_package_write after do_package > > + > +do_package_index[depends] += "dpkg-native:do_populate_sysroot apt-native:do_populate_sysroot" > +do_package_index[recrdeptask] += "package_update_index_deb" > + > +do_package_index() { > + set -ex > + package_update_index_deb > + set +ex > +} Why do you continue to include the do_package_index here, it's no longer needed with the setting of do_package_index[recrdeptask], this is true below also. > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass > index 3c2472b..2a2d9c3 100644 > --- a/meta/classes/package_ipk.bbclass > +++ b/meta/classes/package_ipk.bbclass > @@ -386,3 +386,15 @@ python do_package_write_ipk () { > } > do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}" > addtask package_write_ipk before do_package_write after do_package > + > + > +do_package_index[depends] += "opkg-utils-native:do_populate_sysroot" > +do_package_index[depends] += "opkg-native:do_populate_sysroot" > +do_package_index[recrdeptask] += "package_update_index_ipk" > + > +do_package_index() { > + set -ex > + package_update_index_ipk > + set +ex > +} The do_package_index() function is not needed any longer here > + > diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass > index 70170d1..d9470d6 100644 > --- a/meta/classes/package_rpm.bbclass > +++ b/meta/classes/package_rpm.bbclass > @@ -814,3 +814,10 @@ python do_package_write_rpm () { > do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}" > addtask package_write_rpm before do_package_write after do_package > > +do_package_index[depends] += "createrepo-native:do_populate_sysroot" > +do_package_index() { > + set -ex > + package_update_index_rpm > + createrepo "${DEPLOY_DIR_RPM}" > + set +ex > +} Nor this do_package_index() function, why no do_package_index[recrdeptask] setting in RPM? If you need a special function here them call it do_package_index_rpm() and set the do_package_index[recrdeptask] to that function. > diff --git a/meta/recipes-core/meta/package-index.bb b/meta/recipes-core/meta/package-index.bb > index 3c642cb..3f1caea 100644 > --- a/meta/recipes-core/meta/package-index.bb > +++ b/meta/recipes-core/meta/package-index.bb > @@ -19,14 +19,6 @@ do_package_write_deb[noexec] = "1" > do_populate_sysroot[noexec] = "1" > > do_package_index[nostamp] = "1" > -do_package_index[dirs] = "${DEPLOY_DIR_IPK}" Does the dirs need to be saved or moved to the packagers? > -do_package_index[depends] += "opkg-utils-native:do_populate_sysroot" > -do_package_index[depends] += "opkg-native:do_populate_sysroot" > > -do_package_index() { > - set -ex > - package_update_index_ipk > - set +ex > -} > addtask do_package_index before do_build > EXCLUDE_FROM_WORLD = "1"
Saul Wold wrote: > On 05/13/2011 02:53 AM, Dexuan Cui wrote: >> From: Dexuan Cui<dexuan.cui@intel.com> >> >> diff --git a/meta/classes/package_deb.bbclass >> b/meta/classes/package_deb.bbclass >> index 4faeb4a..4cc0b69 100644 >> --- a/meta/classes/package_deb.bbclass >> +++ b/meta/classes/package_deb.bbclass >> @@ -431,3 +431,12 @@ python do_package_write_deb () { >> do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}" >> addtask package_write_deb before do_package_write after do_package >> >> + >> +do_package_index[depends] += "dpkg-native:do_populate_sysroot >> apt-native:do_populate_sysroot" +do_package_index[recrdeptask] += >> "package_update_index_deb" + +do_package_index() { >> + set -ex >> + package_update_index_deb >> + set +ex >> +} > > Why do you continue to include the do_package_index here, it's no > longer needed with the setting of do_package_index[recrdeptask], this > is true below also. Sorry, I didn't use the recrdeptask flag and I guess I still don't quite understand it. Here removing the do_package_index() I will get such an ERROR: ERROR: Task do_package_index from /distro/dcui/pc1/meta/recipes-core/meta/package-index.bb seems to be empty?!########################## | ETA: 00:00:00 ERROR: Error parsing /distro/dcui/pc1/meta/recipes-core/meta/package-index.bb: md5() argument 1 must be string or read-only buffer, not None | ETA: 00:00:00 About the recrdeptask flag: "These are specified with the 'recrdeptask' flag and is used signify the task(s) of each RDEPENDS which must have completed before that task can be executed. It applies recursively so also, the RDEPENDS of each item in the original RDEPENDS must be met and so on. It also runs all DEPENDS first too". I even added RDEPENDS += " package-index" in package-index.bb, but no change. I'm trying to figure this out. Please point out my issue once this is obvious to you. :-) BTW: using the recrdeptask method, we have no chance to do "set -ex" and as a result we lose some debug info in the log file. Is this ok? >> + >> diff --git a/meta/classes/package_rpm.bbclass >> b/meta/classes/package_rpm.bbclass >> index 70170d1..d9470d6 100644 >> --- a/meta/classes/package_rpm.bbclass >> +++ b/meta/classes/package_rpm.bbclass >> @@ -814,3 +814,10 @@ python do_package_write_rpm () { >> do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}" >> addtask package_write_rpm before do_package_write after do_package >> >> +do_package_index[depends] += >> "createrepo-native:do_populate_sysroot" +do_package_index() { + >> set -ex + package_update_index_rpm >> + createrepo "${DEPLOY_DIR_RPM}" >> + set +ex >> +} > > Nor this do_package_index() function, why no > do_package_index[recrdeptask] setting in RPM? If you need a special > function here them call it do_package_index_rpm() and set the > do_package_index[recrdeptask] to that function. Yes, I should have added a new function including package_update_index_rpm and createrepo "${DEPLOY_DIR_RPM}". and set do_package_index[recrdeptask] to it. Thanks, -- Dexuan
Hi Dexuan,
When I read Saul's original emails I thought this approach would work
although it wasn't the first one that came to my mind.
Looking at the code you have, the problem is you have nowhere you can
call the "addtask" for the individual tasks and we don't want to the
overhead system wide for every recipe as it makes no sense.
I'd therefore like to suggest we tweak this approach a little. Each
package_xxx.bbclass defines a function which updates the index for that
packaging backend:
package_ipk.bbclass:
PACKAGEINDEXES += "package_index_ipk();"
PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot"
PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot"
package_index_ipk[dirs] = "${DEPLOY_DIR_IPK}"
package_index_ipk() {
set -ex
package_update_index_ipk
set +ex
}
The key is the addition to the PACKAGEINDEXES variable and its
dependencies in PACKAGEINDEXDEPS.
In package-index.bb we can then just have:
do_package_index[depends] += "${PACKAGEINDEXDEPS}"
do_package_index() {
${PACKAGEINDEXES}
}
and these variables should always have the things we need in them
depending on the package backends available.
Ideally we should cover the case there is no package backend enabled and
just return too rather than error (or bbwarn "No package backends
enabled, doing nothing").
Cheers,
Richard
Richard Purdie wrote: > Hi Dexuan, > > When I read Saul's original emails I thought this approach would work > although it wasn't the first one that came to my mind. > > Looking at the code you have, the problem is you have nowhere you can > call the "addtask" for the individual tasks and we don't want to the > overhead system wide for every recipe as it makes no sense. Yes, I realized this later. > I'd therefore like to suggest we tweak this approach a little. Each > package_xxx.bbclass defines a function which updates the index for > that packaging backend: I agree. I've actually made a patch in branch dcui/package-index, but the below implementation you suggested is obviously much better! :-) Thank you very much for the detailed suggestion! > package_ipk.bbclass: > > PACKAGEINDEXES += "package_index_ipk();" > PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot" > PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot" > package_index_ipk[dirs] = "${DEPLOY_DIR_IPK}" > > package_index_ipk() { > set -ex > package_update_index_ipk > set +ex > } > > The key is the addition to the PACKAGEINDEXES variable and its > dependencies in PACKAGEINDEXDEPS. > > In package-index.bb we can then just have: > > do_package_index[depends] += "${PACKAGEINDEXDEPS}" > > do_package_index() { > ${PACKAGEINDEXES} > } > > and these variables should always have the things we need in them > depending on the package backends available. > > Ideally we should cover the case there is no package backend enabled > and just return too rather than error (or bbwarn "No package backends > enabled, doing nothing"). Oh, I didn't realize this... I'll remember to handle this, too. Thanks, -- Dexuan
Patch
diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass index 4faeb4a..4cc0b69 100644 --- a/meta/classes/package_deb.bbclass +++ b/meta/classes/package_deb.bbclass @@ -431,3 +431,12 @@ python do_package_write_deb () { do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}" addtask package_write_deb before do_package_write after do_package + +do_package_index[depends] += "dpkg-native:do_populate_sysroot apt-native:do_populate_sysroot" +do_package_index[recrdeptask] += "package_update_index_deb" + +do_package_index() { + set -ex + package_update_index_deb + set +ex +} diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass index 3c2472b..2a2d9c3 100644 --- a/meta/classes/package_ipk.bbclass +++ b/meta/classes/package_ipk.bbclass @@ -386,3 +386,15 @@ python do_package_write_ipk () { } do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}" addtask package_write_ipk before do_package_write after do_package + + +do_package_index[depends] += "opkg-utils-native:do_populate_sysroot" +do_package_index[depends] += "opkg-native:do_populate_sysroot" +do_package_index[recrdeptask] += "package_update_index_ipk" + +do_package_index() { + set -ex + package_update_index_ipk + set +ex +} + diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass index 70170d1..d9470d6 100644 --- a/meta/classes/package_rpm.bbclass +++ b/meta/classes/package_rpm.bbclass @@ -814,3 +814,10 @@ python do_package_write_rpm () { do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}" addtask package_write_rpm before do_package_write after do_package +do_package_index[depends] += "createrepo-native:do_populate_sysroot" +do_package_index() { + set -ex + package_update_index_rpm + createrepo "${DEPLOY_DIR_RPM}" + set +ex +} diff --git a/meta/recipes-core/meta/package-index.bb b/meta/recipes-core/meta/package-index.bb index 3c642cb..3f1caea 100644 --- a/meta/recipes-core/meta/package-index.bb +++ b/meta/recipes-core/meta/package-index.bb @@ -19,14 +19,6 @@ do_package_write_deb[noexec] = "1" do_populate_sysroot[noexec] = "1" do_package_index[nostamp] = "1" -do_package_index[dirs] = "${DEPLOY_DIR_IPK}" -do_package_index[depends] += "opkg-utils-native:do_populate_sysroot" -do_package_index[depends] += "opkg-native:do_populate_sysroot" -do_package_index() { - set -ex - package_update_index_ipk - set +ex -} addtask do_package_index before do_build EXCLUDE_FROM_WORLD = "1"