| Submitter | Saul Wold |
|---|---|
| Date | Oct. 18, 2012, 7:25 p.m. |
| Message ID | <1350588305-16854-1-git-send-email-sgw@linux.intel.com> |
| Download | mbox | patch |
| Permalink | /patch/38289/ |
| State | New |
| Headers | show |
Comments
On Thu, Oct 18, 2012 at 12:25:05PM -0700, Saul Wold wrote: > Present the manifest file that contains the matches for > files being installed to a location that already contains > that file. This will help to determine which is the correct > recipe to fix when this occurs. > > [YOCTO #3191] Can we do the same when removing files from sysroot? To catch errors like this: http://git.openembedded.org/openembedded-core/commit/?id=6b12d4cd39bacb087654b59e25f5052a4e839b26 12:40:15 < JaMa> it depends on order of tasks 12:41:27 < JaMa> gettext-native depends on gettext-minimal-native, so gettext-minimal-native installs own config.rpath to sysroot (when gettext-native "owns" it too) 12:41:55 < JaMa> then gettext-native is built and before populting sysroot it removes own files there (including config.rpath) 12:43:45 < JaMa> and because sstate assumes that every file in sysroot is provided by exactly one recipe, gettext-minimal-native looses his config.rpath 12:44:49 < JaMa> now there is at least warning when 2 recipes are installing the same file to sysroot, but that doesn't resolve this issues when some file is "moved" between recipes 12:47:30 < JaMa> because gettext-minimal-native is only recipe installing it 12:47:39 < JaMa> and gettext-native is only removing it If we check manifests from other recipes before removing files we can show warning that gettext-native is trying to remove config.rpath now provided by gettext-minimal-native and keep config.rpath in sysroot. Now everybody doing incremental builds needs to call bitbake -c cleansstate gettext-minimal-native Cheers,
On Fri, 2012-10-19 at 12:55 +0200, Martin Jansa wrote: > On Thu, Oct 18, 2012 at 12:25:05PM -0700, Saul Wold wrote: > > Present the manifest file that contains the matches for > > files being installed to a location that already contains > > that file. This will help to determine which is the correct > > recipe to fix when this occurs. > > > > [YOCTO #3191] > > Can we do the same when removing files from sysroot? > > To catch errors like this: > http://git.openembedded.org/openembedded-core/commit/?id=6b12d4cd39bacb087654b59e25f5052a4e839b26 > > 12:40:15 < JaMa> it depends on order of tasks > 12:41:27 < JaMa> gettext-native depends on gettext-minimal-native, so gettext-minimal-native installs own config.rpath to sysroot (when gettext-native "owns" > it too) > 12:41:55 < JaMa> then gettext-native is built and before populting sysroot it removes own files there (including config.rpath) > 12:43:45 < JaMa> and because sstate assumes that every file in sysroot is provided by exactly one recipe, gettext-minimal-native looses his config.rpath > 12:44:49 < JaMa> now there is at least warning when 2 recipes are installing the same file to sysroot, but that doesn't resolve this issues when some file is > "moved" between recipes > 12:47:30 < JaMa> because gettext-minimal-native is only recipe installing it > 12:47:39 < JaMa> and gettext-native is only removing it > > If we check manifests from other recipes before removing files we can show warning that > gettext-native is trying to remove config.rpath now provided by gettext-minimal-native > and keep config.rpath in sysroot. The plan is to make it a hard error if something installs a file that already exists. This issue will be resolved when that happens and I want to get there sooner than later. Running the check at sysroot clean time is therefore going to complicate the code a lot and damage performance with no real end need... > Now everybody doing incremental builds needs to call > bitbake -c cleansstate gettext-minimal-native -c clean should be sufficient. I wish people would use appropriately sized hammers and this is why I never wanted cleansstate in the first place :(. Cheers, Richard
On Fri, Oct 19, 2012 at 12:07:05PM +0100, Richard Purdie wrote: > On Fri, 2012-10-19 at 12:55 +0200, Martin Jansa wrote: > > On Thu, Oct 18, 2012 at 12:25:05PM -0700, Saul Wold wrote: > > > Present the manifest file that contains the matches for > > > files being installed to a location that already contains > > > that file. This will help to determine which is the correct > > > recipe to fix when this occurs. > > > > > > [YOCTO #3191] > > > > Can we do the same when removing files from sysroot? > > > > To catch errors like this: > > http://git.openembedded.org/openembedded-core/commit/?id=6b12d4cd39bacb087654b59e25f5052a4e839b26 > > > > 12:40:15 < JaMa> it depends on order of tasks > > 12:41:27 < JaMa> gettext-native depends on gettext-minimal-native, so gettext-minimal-native installs own config.rpath to sysroot (when gettext-native "owns" > > it too) > > 12:41:55 < JaMa> then gettext-native is built and before populting sysroot it removes own files there (including config.rpath) > > 12:43:45 < JaMa> and because sstate assumes that every file in sysroot is provided by exactly one recipe, gettext-minimal-native looses his config.rpath > > 12:44:49 < JaMa> now there is at least warning when 2 recipes are installing the same file to sysroot, but that doesn't resolve this issues when some file is > > "moved" between recipes > > 12:47:30 < JaMa> because gettext-minimal-native is only recipe installing it > > 12:47:39 < JaMa> and gettext-native is only removing it > > > > If we check manifests from other recipes before removing files we can show warning that > > gettext-native is trying to remove config.rpath now provided by gettext-minimal-native > > and keep config.rpath in sysroot. > > The plan is to make it a hard error if something installs a file that > already exists. This issue will be resolved when that happens and I want > to get there sooner than later. For new recipes yes, but that does not resolve this case when one file is moved from one recipe to another. > Running the check at sysroot clean time is therefore going to complicate > the code a lot and damage performance with no real end need... > > > Now everybody doing incremental builds needs to call > > bitbake -c cleansstate gettext-minimal-native > > -c clean should be sufficient. I wish people would use appropriately > sized hammers and this is why I never wanted cleansstate in the first > place :(.
On Thu, Oct 18, 2012 at 12:25 PM, Saul Wold <sgw@linux.intel.com> wrote: > # Avoid docbook/sgml catalog warnings for now > -SSTATE_DUPWHITELIST += "${STAGING_ETCDIR_NATIVE}/sgml ${STAGING_DATADIR_NATIVE}/sgml" > +#SSTATE_DUPWHITELIST += "${STAGING_ETCDIR_NATIVE}/sgml ${STAGING_DATADIR_NATIVE}/sgml" Was this block of the patch intended? It's not mentioned in the commit message, that I can see.
Patch
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass index d2a120b..140dd59 100644 --- a/meta/classes/sstate.bbclass +++ b/meta/classes/sstate.bbclass @@ -27,7 +27,7 @@ SSTATE_DUPWHITELIST += "${STAGING_LIBDIR_NATIVE}/${MULTIMACH_TARGET_SYS} ${STAGI # Also avoid python issues until we fix the python recipe SSTATE_DUPWHITELIST += "${STAGING_LIBDIR}/python2.7/config/Makefile ${STAGING_LIBDIR}/libpython2.7 ${STAGING_INCDIR}/python2.7/pyconfig.h" # Avoid docbook/sgml catalog warnings for now -SSTATE_DUPWHITELIST += "${STAGING_ETCDIR_NATIVE}/sgml ${STAGING_DATADIR_NATIVE}/sgml" +#SSTATE_DUPWHITELIST += "${STAGING_ETCDIR_NATIVE}/sgml ${STAGING_DATADIR_NATIVE}/sgml" SSTATE_SCAN_FILES ?= "*.la *-config *_config" SSTATE_SCAN_CMD ?= 'find ${SSTATE_BUILDDIR} \( -name "${@"\" -o -name \"".join(d.getVar("SSTATE_SCAN_FILES", True).split())}" \) -type f' @@ -120,6 +120,7 @@ def sstate_add(ss, source, dest, d): def sstate_install(ss, d): import oe.path + import subprocess sharedfiles = [] shareddirs = [] @@ -164,8 +165,12 @@ def sstate_install(ss, d): break if realmatch: match.append(f) + sstate_search_cmd = "grep -rl %s %s | sed -e 's:^.*/::' -e 's:\.populate-sysroot::'" % (f, d.expand("${SSTATE_MANIFESTS}")) + search_output = subprocess.check_output(sstate_search_cmd, stderr=subprocess.STDOUT, shell=True) + if search_output != None: + match.append("Matched in %s" % search_output.rstrip()) if match: - bb.warn("The recipe %s is trying to install files into a shared area when those files already exist. Those files are:\n %s" % (d.getVar("PN", True), "\n ".join(match))) + bb.warn("The recipe %s is trying to install files into a shared area when those files already exist. Those files and their manifest location are:\n %s\nPlease verify which package should provide the above files." % (d.getVar('PN', True), "\n ".join(match))) # Write out the manifest f = open(manifest, "w")
Present the manifest file that contains the matches for files being installed to a location that already contains that file. This will help to determine which is the correct recipe to fix when this occurs. [YOCTO #3191] Signed-off-by: Saul Wold <sgw@linux.intel.com> --- meta/classes/sstate.bbclass | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)