Patchwork [RFC,1/2] opkg svn: Add the --force-arch option

login
register
mail settings
Submitter Robert Yang
Date Sept. 5, 2012, 9:31 a.m.
Message ID <4a36b4005fc384ac817023873b2febc854c4415d.1346837268.git.liezhi.yang@windriver.com>
Download mbox | patch
Permalink /patch/35917/
State New
Headers show

Comments

Robert Yang - Sept. 5, 2012, 9:31 a.m.
If there are more than one candidate which has the same pkg name in the
candidate list, for example, the same pkg with different versions, then
it will use the last one which is the highest version in the list

Add the "--force-arch" (just like the --force-downgrade) option to let
it use the higher arch priority package when enabled. the default is no.

[YOCTO #2575]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-devtools/opkg/opkg/force-arch.patch |  103 ++++++++++++++++++++++
 meta/recipes-devtools/opkg/opkg_svn.bb           |    3 +-
 2 files changed, 105 insertions(+), 1 deletions(-)
 create mode 100644 meta/recipes-devtools/opkg/opkg/force-arch.patch
Phil Blundell - Sept. 8, 2012, 8:08 p.m.
On Wed, 2012-09-05 at 17:31 +0800, Robert Yang wrote:
> If there are more than one candidate which has the same pkg name in the
> candidate list, for example, the same pkg with different versions, then
> it will use the last one which is the highest version in the list
> 
> Add the "--force-arch" (just like the --force-downgrade) option to let
> it use the higher arch priority package when enabled. the default is no.

This seems like a reasonable feature to add to opkg, though I don't
think "--force-arch" is a very good name for the switch.  (That name
implies to me "allow the package to be installed even if the
architecture seems to be incompatible", which is quite different to what
your option does.) 

However, regarding the underlying problem:

a) for people who are not using O_P_M, it's becoming apparent that the
whole concept of using opkg (or rpm, or any other external package
manager) for rootfs construction is more of a liability than an asset
because bitbake has more knowledge about which particular binaries ought
to be installed.  For those use-cases, it might be better to think in
terms of abolishing opkg altogether which would solve this problem and a
variety of others.

b) for people who _are_ using O_P_M, specifying --force-arch during
rootfs construction is all very well but they might still lose during a
subsequent "opkg upgrade".  So for these use cases, it seems as though
something a bit more sticky might be required.

p.
Paul Eggleton - Sept. 8, 2012, 8:40 p.m.
On Saturday 08 September 2012 21:08:55 Phil Blundell wrote:
> a) for people who are not using O_P_M, it's becoming apparent that the
> whole concept of using opkg (or rpm, or any other external package
> manager) for rootfs construction is more of a liability than an asset
> because bitbake has more knowledge about which particular binaries ought
> to be installed.  For those use-cases, it might be better to think in
> terms of abolishing opkg altogether which would solve this problem and a
> variety of others.

On the other hand, using the package manager for rootfs construction for those 
that *are* using online package management allows us to have at least some 
confidence that a rootfs produced directly from the metadata and one produced 
by on-system upgrades will be the same; you can also have package management 
on in one image and off in another (or change it on the fly) and expect to have 
the same contents, apart from the package manager being removed. The potential 
for subtle differences in behaviour is too great to go down the path of 
implementing it ourselves IMO, not to mention the extra code paths to 
maintain.

> b) for people who _are_ using O_P_M, specifying --force-arch during
> rootfs construction is all very well but they might still lose during a
> subsequent "opkg upgrade".  So for these use cases, it seems as though
> something a bit more sticky might be required.

In terms of a proper solution I agree with this - opkg needs to handle the 
package architecture configuration internally rather than us overriding it at 
rootfs construction time. If you're advocating spending effort implementing 
something I think that's where it should be spent.

Cheers,
Paul
Phil Blundell - Sept. 8, 2012, 9:18 p.m.
On Sat, 2012-09-08 at 21:40 +0100, Paul Eggleton wrote:
> On Saturday 08 September 2012 21:08:55 Phil Blundell wrote:
> > a) for people who are not using O_P_M, it's becoming apparent that the
> > whole concept of using opkg (or rpm, or any other external package
> > manager) for rootfs construction is more of a liability than an asset
> > because bitbake has more knowledge about which particular binaries ought
> > to be installed.  For those use-cases, it might be better to think in
> > terms of abolishing opkg altogether which would solve this problem and a
> > variety of others.
> 
> On the other hand, using the package manager for rootfs construction for those 
> that *are* using online package management allows us to have at least some 
> confidence that a rootfs produced directly from the metadata and one produced 
> by on-system upgrades will be the same; you can also have package management 
> on in one image and off in another (or change it on the fly) and expect to have 
> the same contents, apart from the package manager being removed. The potential 
> for subtle differences in behaviour is too great to go down the path of 
> implementing it ourselves IMO, not to mention the extra code paths to 
> maintain.

I agree that for the folks who do want O_P_M, using the same package
manager for rootfs construction that they will use online is the only
sensible way to build images.  But it'd be interesting to consider what
proportion of the oe-core userbase genuinely requires O_P_M for their
usecase -- I suspect it's probably a minority nowadays and, personally,
I would consider it a fairly bad idea to ship a product which relied on
opkg-based O_P_M in anything other than a trivial way.

I've long been tempted to implement a pure bitbake-based rootfs
construction system for micro and get rid of opkg altogether.  This
would:

a) improve build time, since it would eliminate the (relatively slow)
opkg-make-index step.  In principle I think one could eliminate the
entire package_write_xx step as well since the necessary data for
installation can be recovered directly from sstate_xxx_package.tgz.

b) not make package choice subject to opkg's whims, and ensure that the
runtime dependency resolution matches what bitbake had decided to do
during the build

c) avoid a class of problems to do with stale data in deploy/ipk if the
package versions in the metadata go backwards (e.g. checking out an
older version or a different branch).

As you say, there would be a certain amount of work involved in making
this happen, but on the other hand bitbake does already carry out most
of the necessary dependency analysis in order to do the right thing with
statically-specified RDEPENDS in recipes.

p.
Robert Yang - Sept. 11, 2012, 10:30 a.m.
Hi All,

Thank you very much for your suggestions, do we have a final solution
or work around please?

If we refer to the rpm, it doesn't care about the package's version, it
just selects the newest build package, I had applied a patch to make it
respect to the arch before.

It seems that we have no good solution for the PREFERRED_VERSION
pkg during the package installation, what does the package management
system knows is the arch's priority, the PREFERRED_VERSION pkg always
has a higher priority than others in the feed. So maybe respect to the
arch is the only way that we have current.

I'd like to submit such a patch if you are OK with it:

Let the arch priority win the higher version is the default way for opkg,
and add an option (--select-higher-version) for it to make the higher
version win the arch priority, so that the user can have another choice.

// Robert

On 09/09/2012 04:40 AM, Paul Eggleton wrote:
> On Saturday 08 September 2012 21:08:55 Phil Blundell wrote:
>> a) for people who are not using O_P_M, it's becoming apparent that the
>> whole concept of using opkg (or rpm, or any other external package
>> manager) for rootfs construction is more of a liability than an asset
>> because bitbake has more knowledge about which particular binaries ought
>> to be installed.  For those use-cases, it might be better to think in
>> terms of abolishing opkg altogether which would solve this problem and a
>> variety of others.
>
> On the other hand, using the package manager for rootfs construction for those
> that *are* using online package management allows us to have at least some
> confidence that a rootfs produced directly from the metadata and one produced
> by on-system upgrades will be the same; you can also have package management
> on in one image and off in another (or change it on the fly) and expect to have
> the same contents, apart from the package manager being removed. The potential
> for subtle differences in behaviour is too great to go down the path of
> implementing it ourselves IMO, not to mention the extra code paths to
> maintain.
>
>> b) for people who _are_ using O_P_M, specifying --force-arch during
>> rootfs construction is all very well but they might still lose during a
>> subsequent "opkg upgrade".  So for these use cases, it seems as though
>> something a bit more sticky might be required.
>
> In terms of a proper solution I agree with this - opkg needs to handle the
> package architecture configuration internally rather than us overriding it at
> rootfs construction time. If you're advocating spending effort implementing
> something I think that's where it should be spent.
>
> Cheers,
> Paul
>

Patch

diff --git a/meta/recipes-devtools/opkg/opkg/force-arch.patch b/meta/recipes-devtools/opkg/opkg/force-arch.patch
new file mode 100644
index 0000000..4f4e9c9
--- /dev/null
+++ b/meta/recipes-devtools/opkg/opkg/force-arch.patch
@@ -0,0 +1,103 @@ 
+Add the --force-arch option
+
+If there are more than one candidate which has the same pkg name in the
+candidate list, for example, the same pkg with different versions, then
+it will use the last one which is the highest version in the list
+
+Add the "--force-arch" (just like the --force-downgrade) option to let
+it use the higher arch priority package when enabled. the default is no.
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
+---
+ libopkg/opkg_conf.h |    1 +
+ libopkg/pkg_hash.c  |   18 +++++++++++++++---
+ src/opkg-cl.c       |    8 ++++++++
+ 3 files changed, 24 insertions(+), 3 deletions(-)
+
+diff --git a/libopkg/opkg_conf.h b/libopkg/opkg_conf.h
+index 3a60bc5..c425219 100644
+--- a/libopkg/opkg_conf.h
++++ b/libopkg/opkg_conf.h
+@@ -77,6 +77,7 @@ struct opkg_conf
+      int force_removal_of_essential_packages;
+      int force_postinstall;
+      int force_remove;
++     int force_arch;
+      int check_signature;
+      int nodeps; /* do not follow dependencies */
+      char *offline_root;
+diff --git a/libopkg/pkg_hash.c b/libopkg/pkg_hash.c
+index a99cf6b..63a67d6 100644
+--- a/libopkg/pkg_hash.c
++++ b/libopkg/pkg_hash.c
+@@ -376,10 +376,22 @@ pkg_hash_fetch_best_installation_candidate(abstract_pkg_t *apkg,
+           if (constraint_fcn(matching, cdata)) {
+              opkg_msg(DEBUG, "Candidate: %s %s.\n",
+ 			     matching->name, matching->version) ;
+-             good_pkg_by_name = matching;
+ 	     /* It has been provided by hand, so it is what user want */
+-             if (matching->provided_by_hand == 1)
+-                break;
++             if (matching->provided_by_hand == 1) {
++                 good_pkg_by_name = matching;
++                 break;
++             }
++             /* Respect to the arch priorities when given alternatives */
++             if (good_pkg_by_name && conf->force_arch) {
++                 if (matching->arch_priority >= good_pkg_by_name->arch_priority) {
++                     good_pkg_by_name = matching;
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         matching->name, matching->version) ;
++                 } else
++                     opkg_msg(DEBUG, "%s %s wins by priority.\n",
++                         good_pkg_by_name->name, good_pkg_by_name->version) ;
++             } else
++                 good_pkg_by_name = matching;
+           }
+      }
+ 
+diff --git a/src/opkg-cl.c b/src/opkg-cl.c
+index 1b927e5..4d3995f 100644
+--- a/src/opkg-cl.c
++++ b/src/opkg-cl.c
+@@ -42,6 +42,7 @@ enum {
+ 	ARGS_OPT_FORCE_SPACE,
+ 	ARGS_OPT_FORCE_POSTINSTALL,
+ 	ARGS_OPT_FORCE_REMOVE,
++	ARGS_OPT_FORCE_ARCH,
+ 	ARGS_OPT_ADD_ARCH,
+ 	ARGS_OPT_ADD_DEST,
+ 	ARGS_OPT_NOACTION,
+@@ -83,6 +84,8 @@ static struct option long_options[] = {
+ 	{"force_postinstall", 0, 0, ARGS_OPT_FORCE_POSTINSTALL},
+ 	{"force-remove", 0, 0, ARGS_OPT_FORCE_REMOVE},
+ 	{"force_remove", 0, 0, ARGS_OPT_FORCE_REMOVE},
++	{"force-arch", 0, 0, ARGS_OPT_FORCE_ARCH},
++	{"force_arch", 0, 0, ARGS_OPT_FORCE_ARCH},
+ 	{"noaction", 0, 0, ARGS_OPT_NOACTION},
+ 	{"download-only", 0, 0, ARGS_OPT_DOWNLOAD_ONLY},
+ 	{"nodeps", 0, 0, ARGS_OPT_NODEPS},
+@@ -173,6 +176,9 @@ args_parse(int argc, char *argv[])
+ 		case ARGS_OPT_FORCE_REMOVE:
+ 			conf->force_remove = 1;
+ 			break;
++		case ARGS_OPT_FORCE_ARCH:
++			conf->force_arch= 1;
++			break;
+ 		case ARGS_OPT_NODEPS:
+ 			conf->nodeps = 1;
+ 			break;
+@@ -281,6 +287,8 @@ usage()
+ 	printf("\t--force-space		Disable free space checks\n");
+ 	printf("\t--force-postinstall	Run postinstall scripts even in offline mode\n");
+ 	printf("\t--force-remove	Remove package even if prerm script fails\n");
++	printf("\t--force-arch		Use the higher arch priority package rather than the\n");
++	printf("\t			higher version one if more than one candidate is found.\n");
+ 	printf("\t--noaction		No action -- test only\n");
+ 	printf("\t--download-only	No action -- download only\n");
+ 	printf("\t--nodeps		Do not follow dependencies\n");
+-- 
+1.7.1
+
diff --git a/meta/recipes-devtools/opkg/opkg_svn.bb b/meta/recipes-devtools/opkg/opkg_svn.bb
index a0667ac..4922d2a 100644
--- a/meta/recipes-devtools/opkg/opkg_svn.bb
+++ b/meta/recipes-devtools/opkg/opkg_svn.bb
@@ -7,6 +7,7 @@  SRC_URI = "svn://opkg.googlecode.com/svn;module=trunk;protocol=http \
            file://offline_postinstall.patch\
            file://track_parents.patch \
            file://conf_override.patch \
+           file://force-arch.patch \
 "
 
 S = "${WORKDIR}/trunk"
@@ -14,4 +15,4 @@  S = "${WORKDIR}/trunk"
 SRCREV = "633"
 PV = "0.1.8+svnr${SRCPV}"
 
-PR = "${INC_PR}.1"
+PR = "${INC_PR}.2"