[V2,2/2] sstate.bbclass: set SSTATE_EXTRAPATHWILDCARD explicitly

Submitted by Robert Yang on Nov. 23, 2018, 2:37 a.m. | Patch ID: 156558

Details

Message ID 1f5301edbaa5868f5acfc104486283225a16d9a9.1542940552.git.liezhi.yang@windriver.com
State New
Headers show

Commit Message

Robert Yang Nov. 23, 2018, 2:37 a.m.
The glob.glob("/sstate/*/*/") is very time consuming, set
SSTATE_EXTRAPATHWILDCARD explicity to avoid that. This can save a lot of time
when there are many sstate files.

For example, I have more than 600,000 sstate files:
* Before
  - Without disk caches
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
    real    4m32.583s
    user    0m5.768s
    sys     0m12.892s

  - With disk caches
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
  real    0m4.111s
  user    0m2.348s
  sys     0m1.756s

* After
  $ time python3 -c 'import glob; glob.glob("/sstate-cache.bak/universal/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
  - Without disk caches:
  real    0m7.928s
  user    0m0.172s
  sys     0m0.124s

  - With disk caches:
  real    0m0.131s
  user    0m0.088s
  sys     0m0.044s

We can see that it saves about 3.8s with disk caches, and saves about 264s
without disk caches.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/sstate.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Patch hide | download patch | download mbox

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 9f059a0..8b48ab4 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -101,7 +101,7 @@  python () {
     if bb.data.inherits_class('native', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross', d):
         d.setVar('SSTATE_EXTRAPATH', "${NATIVELSBSTRING}/")
         d.setVar('BB_HASHFILENAME', "True ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}")
-        d.setVar('SSTATE_EXTRAPATHWILDCARD', "*/")
+        d.setVar('SSTATE_EXTRAPATHWILDCARD', "${NATIVELSBSTRING}/")
 
     unique_tasks = sorted(set((d.getVar('SSTATETASKS') or "").split()))
     d.setVar('SSTATETASKS', " ".join(unique_tasks))