| Submitter | Richard Purdie |
|---|---|
| Date | May 9, 2012, 8:32 p.m. |
| Message ID | <1336595524.2494.77.camel@ted> |
| Download | mbox | patch |
| Permalink | /patch/27403/ |
| State | New |
| Headers | show |
Comments
My rebuild completed in 5 seconds. Thank you very much! I'd rather not fork bitbake locally. Can I expect this patch to show up in git://git.openembedded.org/bitbake sometime soon? --rich On 5/9/12 13:32 , Richard Purdie wrote: > Hi Rich, > > You might like to try the change below as I think it might address the problem. > > Cheers, > > Richard > > bitbake/runqueue: Fix 'full' stamp checking to be more efficient and cache results > > This should fix issues where bitbake would seemingly lock up when checking > certain configurations of stampfiles. > > Signed-off-by: Richard Purdie<richard.purdie@linuxfoundation.org> > --- > diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py > index b870caf..48433be 100644 > --- a/bitbake/lib/bb/runqueue.py > +++ b/bitbake/lib/bb/runqueue.py > @@ -875,7 +875,7 @@ class RunQueue: > bb.msg.fatal("RunQueue", "check_stamps fatal internal error") > return current > > - def check_stamp_task(self, task, taskname = None, recurse = False): > + def check_stamp_task(self, task, taskname = None, recurse = False, cache = {}): > def get_timestamp(f): > try: > if not os.access(f, os.F_OK): > @@ -915,6 +915,9 @@ class RunQueue: > t1 = get_timestamp(stampfile) > for dep in self.rqdata.runq_depends[task]: > if iscurrent: > + if dep in cache: > + iscurrent = cache[dep] > + continue > fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]] > taskname2 = self.rqdata.runq_task[dep] > stampfile2 = bb.build.stampfile(taskname2, self.rqdata.dataCache, fn2) > @@ -931,7 +934,9 @@ class RunQueue: > logger.debug(2, 'Stampfile %s< %s', stampfile, stampfile2) > iscurrent = False > if recurse and iscurrent: > - iscurrent = self.check_stamp_task(dep, recurse=True) > + iscurrent = self.check_stamp_task(dep, recurse=True, cache=cache) > + cache[dep] = iscurrent > + cache[task] = iscurrent > return iscurrent > > def execute_runqueue(self): > > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
On Wed, 2012-05-09 at 16:20 -0700, Rich Pixley wrote: > My rebuild completed in 5 seconds. > > Thank you very much! > > I'd rather not fork bitbake locally. Can I expect this patch to show up > in git://git.openembedded.org/bitbake sometime soon? Yes, I just wanted to confirm we were fixing the right problem... I'll post it to the bitbake list and merge in a day or two assuming no negative feedback. I still wonder if that stamp mode is useful but that is a different discussion. Cheers, Richard
On 5/9/12 16:32 , Richard Purdie wrote: > On Wed, 2012-05-09 at 16:20 -0700, Rich Pixley wrote: >> My rebuild completed in 5 seconds. >> >> Thank you very much! >> >> I'd rather not fork bitbake locally. Can I expect this patch to show up >> in git://git.openembedded.org/bitbake sometime soon? > Yes, I just wanted to confirm we were fixing the right problem... > > I'll post it to the bitbake list and merge in a day or two assuming no > negative feedback. I'm running more thorough tests as I type. > I still wonder if that stamp mode is useful but that is a different > discussion. If I'm understanding your other message, then no, it's probably not. My only concern is that in the A depends on B depends on C case, changing C forces a rebuild of A. I don't really care which mechanism makes that happen. --rich
Patch
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py index b870caf..48433be 100644 --- a/bitbake/lib/bb/runqueue.py +++ b/bitbake/lib/bb/runqueue.py @@ -875,7 +875,7 @@ class RunQueue: bb.msg.fatal("RunQueue", "check_stamps fatal internal error") return current - def check_stamp_task(self, task, taskname = None, recurse = False): + def check_stamp_task(self, task, taskname = None, recurse = False, cache = {}): def get_timestamp(f): try: if not os.access(f, os.F_OK): @@ -915,6 +915,9 @@ class RunQueue: t1 = get_timestamp(stampfile) for dep in self.rqdata.runq_depends[task]: if iscurrent: + if dep in cache: + iscurrent = cache[dep] + continue fn2 = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[dep]] taskname2 = self.rqdata.runq_task[dep] stampfile2 = bb.build.stampfile(taskname2, self.rqdata.dataCache, fn2) @@ -931,7 +934,9 @@ class RunQueue: logger.debug(2, 'Stampfile %s < %s', stampfile, stampfile2) iscurrent = False if recurse and iscurrent: - iscurrent = self.check_stamp_task(dep, recurse=True) + iscurrent = self.check_stamp_task(dep, recurse=True, cache=cache) + cache[dep] = iscurrent + cache[task] = iscurrent return iscurrent def execute_runqueue(self):
Hi Rich, You might like to try the change below as I think it might address the problem. Cheers, Richard bitbake/runqueue: Fix 'full' stamp checking to be more efficient and cache results This should fix issues where bitbake would seemingly lock up when checking certain configurations of stampfiles. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> ---