Patchwork [1/1] ncurses: Disable parallel make

From: Xiaofeng Yan <xiaofeng.yan@windriver.com>

Ncurses failure non-gplv3 build by race issue. So disable parallel \
make when building this package.

Signed-off-by: Xiaofeng Yan <xiaofeng.yan@windriver.com>
---
 meta/recipes-core/ncurses/ncurses.inc |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>
> Ncurses failure non-gplv3 build by race issue. So disable parallel \
> make when building this package.
>
This is not the best approach as you disable PARALLEL_MAKE for both 
non-gplv3 and gplv3 versions.  Further, we want to get rid of 
PARALLLEL_MAKE setting as much as possible, so this patch is not helping 
that.

Did you try running on a large many core machine?  It might help if you 
have some other builds going also to stress the machine.

Sau!

> Signed-off-by: Xiaofeng Yan<xiaofeng.yan@windriver.com>
> ---
>   meta/recipes-core/ncurses/ncurses.inc |    2 ++
>   1 files changed, 2 insertions(+), 0 deletions(-)
>
> diff --git a/meta/recipes-core/ncurses/ncurses.inc b/meta/recipes-core/ncurses/ncurses.inc
> index ae99e2c..b115e46 100644
> --- a/meta/recipes-core/ncurses/ncurses.inc
> +++ b/meta/recipes-core/ncurses/ncurses.inc
> @@ -29,6 +29,8 @@ BUILD_CPPFLAGS += "-D_GNU_SOURCE"
>   # natives don't generally look in base_libdir
>   base_libdir_virtclass-native = "${libdir}"
>
> +PARALLEL_MAKE = ""
> +
>   # Helper function for do_configure to allow multiple configurations
>   # $1 the directory to run configure in
>   # $@ the arguments to pass to configure

On 2012?05?16? 19:02, Saul Wold wrote:
> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>
>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>> make when building this package.
>>
> This is not the best approach as you disable PARALLEL_MAKE for both 
> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1] 
> setting as much as possible, so this patch is not helping that.
>
> Did you try running on a large many core machine? It might help if you 
> have some other builds going also to stress the machine.
>
> Sau!
Thanks for your reply. The most cores I have are eight. I also set 
PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for 
fixing bugs.

Thanks
Yan
>
>> Signed-off-by: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>> ---
>> meta/recipes-core/ncurses/ncurses.inc | 2 ++
>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/meta/recipes-core/ncurses/ncurses.inc 
>> b/meta/recipes-core/ncurses/ncurses.inc
>> index ae99e2c..b115e46 100644
>> --- a/meta/recipes-core/ncurses/ncurses.inc
>> +++ b/meta/recipes-core/ncurses/ncurses.inc
>> @@ -29,6 +29,8 @@ BUILD_CPPFLAGS += "-D_GNU_SOURCE"
>> # natives don't generally look in base_libdir
>> base_libdir_virtclass-native = "${libdir}"
>>
>> +PARALLEL_MAKE = ""
>> +
>> # Helper function for do_configure to allow multiple configurations
>> # $1 the directory to run configure in
>> # $@ the arguments to pass to configure
>

On Wed, May 16, 2012 at 7:01 PM, Xiaofeng Yan
<xiaofeng.yan@windriver.com> wrote:
> Thanks for your reply. The most cores I have are eight. I also set
> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for fixing
> bugs.
>

I think disabling parallel make should be last resort. So yes any
other solution is a good thing to try.

On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
> On 2012?05?16? 19:02, Saul Wold wrote:
>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>
>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>> make when building this package.
>>>
>> This is not the best approach as you disable PARALLEL_MAKE for both 
>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1] 
>> setting as much as possible, so this patch is not helping that.
>>
>> Did you try running on a large many core machine? It might help if you 
>> have some other builds going also to stress the machine.
>>
>> Sau!
> Thanks for your reply. The most cores I have are eight. I also set 
> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for 
> fixing bugs.
> 

Do you have an error file from a failed build (and ideally the failed build directory)?  Having diagnosed many problems like this in the past, it is easiest to look for the failure case and add some sleep statement in the Makefile to get it to trigger every time in the same way. 

The two most common problems are:
  1) autoconf re-runs due to time stamps or partially patched files
  2) a generated file is reported as missing

In the first case it, it will often be some error with a .h missing or some other strange error about a header in the compilation and it is a result of only having a partial file because it is getting regenerated at the time.

In the second case you just find the file's rule in the Makefile and add an if statement in the Make target goal if it is a multi-object rule to look for the problem object and sleep a bit.  I have yet to see a case I couldn't reproduce the results by following the strategy of some forcing some extra delay.  You probably won't have to go to this length, but there was one time I even wrote a C wrapper around a command to add some sleep controlled by an environment variable to prove config.h was getting removed and regenerated.   Example:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(int argc, char *const argv[]) {
   char *lookfor;
   if (argc >= 2) { 
     lookfor = getenv("LOOKFORSLEEP");
     if (lookfor && strcmp(argv[1], lookfor) == 0) {
         if (argc >= 3 && strcmp(argv[2], "config.h") == 0) {
            unlink("config.h");
            printf("Special sleep on command %s\n", lookfor);
            sleep(2);
         }
     }
   }
   execv("/bin/sh", argv);
   return 0;
}


Best of luck,
Jason.

On 2012?05?17? 20:02, Jason Wessel wrote:
> On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
>> On 2012?05?16? 19:02, Saul Wold wrote:
>>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>>
>>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>>> make when building this package.
>>>>
>>> This is not the best approach as you disable PARALLEL_MAKE for both
>>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1]
>>> setting as much as possible, so this patch is not helping that.
>>>
>>> Did you try running on a large many core machine? It might help if you
>>> have some other builds going also to stress the machine.
>>>
>>> Sau!
>> Thanks for your reply. The most cores I have are eight. I also set
>> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for
>> fixing bugs.
>>
> Do you have an error file from a failed build (and ideally the failed build directory)?  Having diagnosed many problems like this in the past, it is easiest to look for the failure case and add some sleep statement in the Makefile to get it to trigger every time in the same way.
>
> The two most common problems are:
>    1) autoconf re-runs due to time stamps or partially patched files
>    2) a generated file is reported as missing
>
> In the first case it, it will often be some error with a .h missing or some other strange error about a header in the compilation and it is a result of only having a partial file because it is getting regenerated at the time.
>
> In the second case you just find the file's rule in the Makefile and add an if statement in the Make target goal if it is a multi-object rule to look for the problem object and sleep a bit.  I have yet to see a case I couldn't reproduce the results by following the strategy of some forcing some extra delay.  You probably won't have to go to this length, but there was one time I even wrote a C wrapper around a command to add some sleep controlled by an environment variable to prove config.h was getting removed and regenerated.   Example:
>
> #include<stdio.h>
> #include<stdlib.h>
> #include<string.h>
> #include<unistd.h>
>
> int main(int argc, char *const argv[]) {
>     char *lookfor;
>     if (argc>= 2) {
>       lookfor = getenv("LOOKFORSLEEP");
>       if (lookfor&&  strcmp(argv[1], lookfor) == 0) {
>           if (argc>= 3&&  strcmp(argv[2], "config.h") == 0) {
>              unlink("config.h");
>              printf("Special sleep on command %s\n", lookfor);
>              sleep(2);
>           }
>       }
>     }
>     execv("/bin/sh", argv);
>     return 0;
> }
>
>
> Best of luck,
> Jason.
>
Hi Jason,

Thank you to share your good experience with me very much.  I will apply 
your methods to fix this bug.

Thanks
Yan

On 2012?05?17? 20:02, Jason Wessel wrote:
> On 05/16/2012 09:01 PM, Xiaofeng Yan wrote:
>> On 2012?05?16? 19:02, Saul Wold wrote:
>>> On 05/16/2012 01:10 PM, xiaofeng.yan@windriver.com wrote:
>>>> From: Xiaofeng Yan<xiaofeng.yan@windriver.com>
>>>>
>>>> Ncurses failure non-gplv3 build by race issue. So disable parallel \
>>>> make when building this package.
>>>>
>>> This is not the best approach as you disable PARALLEL_MAKE for both
>>> non-gplv3 and gplv3 versions. Further, we want to get rid of [M1]
>>> setting as much as possible, so this patch is not helping that.
>>>
>>> Did you try running on a large many core machine? It might help if you
>>> have some other builds going also to stress the machine.
>>>
>>> Sau!
>> Thanks for your reply. The most cores I have are eight. I also set
>> PARALLEL_MAKE=j1000 and 10000. I think I need try to find new way for
>> fixing bugs.
>>
> Do you have an error file from a failed build (and ideally the failed build directory)?  Having diagnosed many problems like this in the past, it is easiest to look for the failure case and add some sleep statement in the Makefile to get it to trigger every time in the same way.
Hi jason,
The failed build information is in *Bug 2298* 
<https://bugzilla.yoctoproject.org/show_bug.cgi?id=2298>.  The error 
appear in the stage of install, not configure and compiling.
Do you any ideas after reading bug information?

Thanks
Yan



> The two most common problems are:
>    1) autoconf re-runs due to time stamps or partially patched files
>    2) a generated file is reported as missing
>
> In the first case it, it will often be some error with a .h missing or some other strange error about a header in the compilation and it is a result of only having a partial file because it is getting regenerated at the time.
>
> In the second case you just find the file's rule in the Makefile and add an if statement in the Make target goal if it is a multi-object rule to look for the problem object and sleep a bit.  I have yet to see a case I couldn't reproduce the results by following the strategy of some forcing some extra delay.  You probably won't have to go to this length, but there was one time I even wrote a C wrapper around a command to add some sleep controlled by an environment variable to prove config.h was getting removed and regenerated.   Example:
>
> #include<stdio.h>
> #include<stdlib.h>
> #include<string.h>
> #include<unistd.h>
>
> int main(int argc, char *const argv[]) {
>     char *lookfor;
>     if (argc>= 2) {
>       lookfor = getenv("LOOKFORSLEEP");
>       if (lookfor&&  strcmp(argv[1], lookfor) == 0) {
>           if (argc>= 3&&  strcmp(argv[2], "config.h") == 0) {
>              unlink("config.h");
>              printf("Special sleep on command %s\n", lookfor);
>              sleep(2);
>           }
>       }
>     }
>     execv("/bin/sh", argv);
>     return 0;
> }
>
>
> Best of luck,
> Jason.
>