Patchwork [1/1] pseudo: Fix problem related to realpath

login
register
mail settings
Submitter Mark Hatle
Date June 9, 2011, 5:15 p.m.
Message ID <f4bf925e5ef24493da42015e1be93d59b85f3a44.1307639649.git.mark.hatle@windriver.com>
Download mbox | patch
Permalink /patch/5611/
State New, archived
Headers show

Comments

Mark Hatle - June 9, 2011, 5:15 p.m.
When pseudo is disabled, certain programs that call realpath may not
work properly.  This was discovered when using the Qt MOC tool when
certain qmake project features are used.

[YOCTO #1150]

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 .../pseudo/pseudo/realpath_fix.patch               |  165 ++++++++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_1.1.1.bb       |    3 +-
 meta/recipes-devtools/pseudo/pseudo_git.bb         |    2 +-
 3 files changed, 168 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
Paul Eggleton - June 10, 2011, 3:02 p.m.
On Thursday 09 June 2011 18:15:45 Mark Hatle wrote:
> --- a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
> +++ b/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
> @@ -1,9 +1,10 @@
>  require pseudo.inc
>  
> -PR = "r0"
> +PR = "r1"

I may be mistaken, but it seems to me that every time we need to rebuild 
pseudo it results in a race when an older version of pseudo is present. We 
handle building pseudo on its own first if it's not present, however if it just 
needs updating we go ahead and rebuild it at the same time as building other 
packages, and if it just happens that pseudo is needed when it's being built 
-> bang. At least that's my assumption given that since the last batch of 
updates, my rebuild of perl-native failed at the same time as pseudo's 
do_compile with a large number of "ERROR: ld.so: object 'libpseudo.so' from 
LD_PRELOAD cannot be preloaded: ignored" messages.

Am I right in that there's currently no mechanism to work around this?

Cheers,
Paul
Mark Hatle - June 10, 2011, 3:14 p.m.
On 6/10/11 10:02 AM, Paul Eggleton wrote:
> On Thursday 09 June 2011 18:15:45 Mark Hatle wrote:
>> --- a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
>> +++ b/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
>> @@ -1,9 +1,10 @@
>>  require pseudo.inc
>>  
>> -PR = "r0"
>> +PR = "r1"
> 
> I may be mistaken, but it seems to me that every time we need to rebuild 
> pseudo it results in a race when an older version of pseudo is present. We 
> handle building pseudo on its own first if it's not present, however if it just 
> needs updating we go ahead and rebuild it at the same time as building other 
> packages, and if it just happens that pseudo is needed when it's being built 
> -> bang. At least that's my assumption given that since the last batch of 
> updates, my rebuild of perl-native failed at the same time as pseudo's 
> do_compile with a large number of "ERROR: ld.so: object 'libpseudo.so' from 
> LD_PRELOAD cannot be preloaded: ignored" messages.
> 
> Am I right in that there's currently no mechanism to work around this?
> 
> Cheers,
> Paul
> 

Theoretically updating pseudo shouldn't happen very often.  But yes, currently
there is an issue that pseudo updates are not caused and cause the pseudo
rebuild.  We can work around this by:

* removing the "pseudodone" file in the build directory

* manually running the "first stage" build:
  BBFETCH2=True PSEUDO_BUILD=1 ../bitbake/bin/bitbake pseudo-native

* manually running "bitbake pseudo-native", (still does it as a second stage,
but usually works)

Likely what we need to do is figure out a (quick) way in the bitbake wrapper to
determine if the pseudo has changed... timestamp match to pseudodone?  That of
course still won't be fool proof.

What has been discussed in the past is adding the capabilities for a "staged"
build to bitbake.  Being able to re-invoke the bitbake process after given
stages [automatically] so that in the end bitbake is in charge of the build
steps.  So far though not much has been done with this idea, as we're really not
sure how practical this is.
Mark Hatle - June 10, 2011, 3:19 p.m.
On 6/10/11 10:14 AM, Mark Hatle wrote:
> On 6/10/11 10:02 AM, Paul Eggleton wrote:
>> On Thursday 09 June 2011 18:15:45 Mark Hatle wrote:
>>> --- a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
>>> +++ b/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
>>> @@ -1,9 +1,10 @@
>>>  require pseudo.inc
>>>  
>>> -PR = "r0"
>>> +PR = "r1"
>>
>> I may be mistaken, but it seems to me that every time we need to rebuild 
>> pseudo it results in a race when an older version of pseudo is present. We 
>> handle building pseudo on its own first if it's not present, however if it just 
>> needs updating we go ahead and rebuild it at the same time as building other 
>> packages, and if it just happens that pseudo is needed when it's being built 
>> -> bang. At least that's my assumption given that since the last batch of 
>> updates, my rebuild of perl-native failed at the same time as pseudo's 
>> do_compile with a large number of "ERROR: ld.so: object 'libpseudo.so' from 
>> LD_PRELOAD cannot be preloaded: ignored" messages.
>>
>> Am I right in that there's currently no mechanism to work around this?
>>
>> Cheers,
>> Paul
>>
> 
> Theoretically updating pseudo shouldn't happen very often.  But yes, currently
> there is an issue that pseudo updates are not caused and cause the pseudo

Wow, too early in the morning I guess..  let me translate that for people who
are awake...

... there is an issue that pseudo updates are not caught by the bitbake wrapper.
 This can cause the pseudo-native rebuild in stage2 to cause problems. We can
work around this by:

> rebuild.  We can work around this by:
> 
> * removing the "pseudodone" file in the build directory
> 
> * manually running the "first stage" build:
>   BBFETCH2=True PSEUDO_BUILD=1 ../bitbake/bin/bitbake pseudo-native
> 
> * manually running "bitbake pseudo-native", (still does it as a second stage,
> but usually works)
> 
> Likely what we need to do is figure out a (quick) way in the bitbake wrapper to
> determine if the pseudo has changed... timestamp match to pseudodone?  That of
> course still won't be fool proof.
> 
> What has been discussed in the past is adding the capabilities for a "staged"
> build to bitbake.  Being able to re-invoke the bitbake process after given
> stages [automatically] so that in the end bitbake is in charge of the build
> steps.  So far though not much has been done with this idea, as we're really not
> sure how practical this is.
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Patch

diff --git a/meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch b/meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
new file mode 100644
index 0000000..7beea51
--- /dev/null
+++ b/meta/recipes-devtools/pseudo/pseudo/realpath_fix.patch
@@ -0,0 +1,165 @@ 
+commit c2f7c5ad8ef0f9c94a2a8382c109c8c6e16c8b18
+Author: Peter Seebach <peter.seebach@windriver.com>
+Date:   Thu Jun 9 10:53:32 2011 -0500
+
+    Fix realpath(name, NULL) when PSEUDO_DISABLED=1
+    
+    On some Linux systems, dlsym("realpath", RTLD_NEXT) prefers
+    for reasons of its own to give a symbol that is also known
+    as old_realpath, which fails and yields EINVAL when called
+    with a null pointer as the second argument.  This can be
+    avoided, on some systems, by using dlvsym() to request
+    the GLIBC_2.3 version of the symbol.
+    
+    The wrapper logic is enhanced to allow for specifying
+    versions, although this currently only works for Linux
+    (Darwin has no dlvsym, apparently?).  The test case is
+    a trivial program which calls realpath(name, NULL) run
+    with PSEUDO_DISABLED=1.
+
+diff --git a/ChangeLog.txt b/ChangeLog.txt
+index 7ffb74a..a2bbb61 100644
+--- a/ChangeLog.txt
++++ b/ChangeLog.txt
+@@ -1,3 +1,8 @@
++2011-06-08:
++	* (seebs) Get the modern realpath from glibc instead of the old
++	  one inexplicably proferred by RTLD_NEXT.  Fixes realpath(path, NULL)
++	  when PSEUDO_DISABLED=1.
++
+ 2011-06-06:
+ 	* (seebs) revise system() handler substantially.  It now
+ 	  pollutes the environment but works.
+diff --git a/makewrappers b/makewrappers
+index 6dcf889..20bbf2b 100755
+--- a/makewrappers
++++ b/makewrappers
+@@ -211,6 +211,7 @@ class Function:
+         self.flags = '0'
+         self.port = port
+         self.directory = ''
++	self.version = 'NULL'
+         # On Darwin, some functions are SECRETLY converted to foo$INODE64
+         # when called.  So we have to look those up for real_*
+         self.inode64 = None
+diff --git a/ports/unix/wrapfuncs.in b/ports/unix/wrapfuncs.in
+index 74bad89..e06e404 100644
+--- a/ports/unix/wrapfuncs.in
++++ b/ports/unix/wrapfuncs.in
+@@ -18,7 +18,7 @@ int lutimes(const char *path, const struct timeval *tv);
+ char *mkdtemp(char *template);
+ char *mktemp(char *template);
+ long pathconf(const char *path, int name);
+-char *realpath(const char *name, char *resolved_name);
++char *realpath(const char *name, char *resolved_name); /* version="GLIBC_2.3" */
+ int remove(const char *path); /* flags=AT_SYMLINK_NOFOLLOW */
+ DIR *opendir(const char *path);
+ char *tempnam(const char *template, const char *pfx);
+diff --git a/pseudo_wrappers.c b/pseudo_wrappers.c
+index 600a918..07a4429 100644
+--- a/pseudo_wrappers.c
++++ b/pseudo_wrappers.c
+@@ -90,6 +90,42 @@ pseudo_reinit_libpseudo(void) {
+ 	_libpseudo_init();
+ }
+ 
++static void
++pseudo_init_one_wrapper(pseudo_function *func) {
++	int (*f)(void);
++	char *e;
++	if (*func->real != NULL) {
++		/* already initialized */
++		return;
++	}
++	dlerror();
++
++#if PSEUDO_PORT_LINUX
++	if (func->version)
++		f = dlvsym(RTLD_NEXT, func->name, func->version);
++	/* fall through to the general case, if that failed */
++	if (!f)
++#endif
++	f = dlsym(RTLD_NEXT, func->name);
++	if (f) {
++		*func->real = f;
++	} else {
++		e = dlerror();
++#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS
++		char *s = func->name;
++		s += strlen(s) - 2;
++		/* *at() don't have to exist */
++		if (!strcmp(s, "at")) {
++			continue;
++		}
++#else
++		if (e != NULL) {
++			pseudo_diag("No real function for %s: %s\n", func->name, e);
++		}
++#endif
++	}
++}
++
+ void
+ pseudo_init_wrappers(void) {
+ 	int i;
+@@ -103,29 +139,7 @@ pseudo_init_wrappers(void) {
+ 	 */
+ 	if (!done) {
+ 		for (i = 0; pseudo_functions[i].name; ++i) {
+-			if (*pseudo_functions[i].real == NULL) {
+-				int (*f)(void);
+-				char *e;
+-				dlerror();
+-				f = dlsym(RTLD_NEXT, pseudo_functions[i].name);
+-				if (f) {
+-					*pseudo_functions[i].real = f;
+-				} else {
+-					e = dlerror();
+-#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS
+-					char *s = pseudo_functions[i].name;
+-					s += strlen(s) - 2;
+-					/* *at() don't have to exist */
+-					if (!strcmp(s, "at")) {
+-						continue;
+-					}
+-#else
+-					if (e != NULL) {
+-						pseudo_diag("No real function for %s: %s\n", pseudo_functions[i].name, e);
+-					}
+-#endif
+-				}
+-			}
++			pseudo_init_one_wrapper(&pseudo_functions[i]);
+ 		}
+ 		done = 1;
+ 	}
+diff --git a/templates/wrapper_table b/templates/wrapper_table
+index 2e79fcd..bb30530 100644
+--- a/templates/wrapper_table
++++ b/templates/wrapper_table
+@@ -4,17 +4,21 @@
+ 
+ /* This file is generated and should not be modified.  See the makewrappers
+  * script if you want to modify this. */
+-static struct {
++typedef struct {
+ 	char *name;		/* the name */
+ 	int (**real)(void);	/* the underlying syscall */
+ 	int (*wrapper)(void);	/* the wrapper from guts/name.c */
+-} pseudo_functions[] = {
++	char *version;		/* the version, if we know and care */
++} pseudo_function;
++
++static pseudo_function pseudo_functions[] = {
+ @body
+ 	{ /* ${comment}; */
+ 		"${name}${maybe_inode64}",
+ 		(int (**)(void)) &real_${name},
+-		(int (*)(void)) wrap_${name}
++		(int (*)(void)) wrap_${name},
++		${version}
+ 	},
+ @footer
+-	{ NULL, NULL, NULL },
++	{ NULL, NULL, NULL, NULL },
+ };
diff --git a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb b/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
index e05fe41..f1c8e63 100644
--- a/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb
@@ -1,9 +1,10 @@ 
 require pseudo.inc
 
-PR = "r0"
+PR = "r1"
 
 SRC_URI = "http://www.yoctoproject.org/downloads/${BPN}/${BPN}-${PV}.tar.bz2 \
            file://oe-config.patch \
+           file://realpath_fix.patch \
            file://static_sqlite.patch"
 
 SRC_URI[md5sum] = "dd59766c17e199fe6144fce8a2c67802"
diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb b/meta/recipes-devtools/pseudo/pseudo_git.bb
index 6505958..c1f0432 100644
--- a/meta/recipes-devtools/pseudo/pseudo_git.bb
+++ b/meta/recipes-devtools/pseudo/pseudo_git.bb
@@ -1,6 +1,6 @@ 
 require pseudo.inc
 
-SRCREV = "1d3e67cb168c3459e67a0b29f071ca30ed17dadc"
+SRCREV = "c2f7c5ad8ef0f9c94a2a8382c109c8c6e16c8b18"
 PV = "1.1.1+git${SRCPV}"
 PR = "r19"