Patchwork sstate: Convert to use ':' as a filename sperator and use SSTATE_SWSPEC globally

login
register
mail settings
Submitter Richard Purdie
Date Dec. 20, 2013, 12:06 p.m.
Message ID <1387541162.6402.91.camel@ted>
Download mbox | patch
Permalink /patch/63623/
State Accepted
Commit e5ee1b8c0d141e56ee67778c7b8af92de185dc01
Headers show

Comments

Richard Purdie - Dec. 20, 2013, 12:06 p.m.
Currently the code has problems differentiating between "gcc-cross" and "gcc-cross-initial"
sstate files. We could add in a ton of special casing but tests show this isn't scaling
well. Using a more unique separator resolves the issue.

The choice of which separator to use is a hard one. We need something which isn't commonly
used in PN, PV, PR, *_OS and *_ARCH which rules out '-', '_' and it needs to work ok with
webservers/http which makes ';' and '%' harder.

The change also sets SSTATE_SWSPEC globally since writing out differently named siginfo
files for the fetch/unpack/patch tasks is a waste of diskspace, the hashes match for
all PN in the majority of cases and if they don't, its not a big issue as the hash is
different. This makes the results from sstate debugging more understandable.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
Martin Jansa - Dec. 20, 2013, 12:33 p.m.
On Fri, Dec 20, 2013 at 12:06:02PM +0000, Richard Purdie wrote:
> Currently the code has problems differentiating between "gcc-cross" and "gcc-cross-initial"
> sstate files. We could add in a ton of special casing but tests show this isn't scaling
> well. Using a more unique separator resolves the issue.
> 
> The choice of which separator to use is a hard one. We need something which isn't commonly
> used in PN, PV, PR, *_OS and *_ARCH which rules out '-', '_' and it needs to work ok with
> webservers/http which makes ';' and '%' harder.
> 
> The change also sets SSTATE_SWSPEC globally since writing out differently named siginfo
> files for the fetch/unpack/patch tasks is a waste of diskspace, the hashes match for
> all PN in the majority of cases and if they don't, its not a big issue as the hash is
> different. This makes the results from sstate debugging more understandable.

Can you mention why :: is sometimes used in SWSPEC?

I guess it's again just for easier differentianting, but maybe it should
be mentioned in commit message.

> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 199a5b3..324d1c1 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -9,8 +9,8 @@ def generate_sstatefn(spec, hash, d):
>      return hash[:2] + "/" + spec + hash
>  
>  SSTATE_PKGARCH    = "${PACKAGE_ARCH}"
> -SSTATE_PKGSPEC    = "sstate-${PN}-${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}-${PV}-${PR}-${SSTATE_PKGARCH}-${SSTATE_VERSION}-"
> -SSTATE_SWSPEC     = ""
> +SSTATE_PKGSPEC    = "sstate:${PN}:${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}:${PV}:${PR}:${SSTATE_PKGARCH}:${SSTATE_VERSION}:"
> +SSTATE_SWSPEC     = "sstate:${BPN}::${PV}:${PR}::${SSTATE_VERSION}:"
>  SSTATE_PKGNAME    = "${SSTATE_EXTRAPATH}${@generate_sstatefn(d.getVar('SSTATE_PKGSPEC', True), d.getVar('BB_TASKHASH', True), d)}"
>  SSTATE_PKG        = "${SSTATE_DIR}/${SSTATE_PKGNAME}"
>  SSTATE_EXTRAPATH   = ""
> diff --git a/meta/recipes-devtools/gcc/gcc-common.inc b/meta/recipes-devtools/gcc/gcc-common.inc
> index e49d724..112e8ce 100644
> --- a/meta/recipes-devtools/gcc/gcc-common.inc
> +++ b/meta/recipes-devtools/gcc/gcc-common.inc
> @@ -95,7 +95,7 @@ do_patch[stamp-base-clean] = "${SSCLEAN}"
>  
>  # SW means Shared Work directory
>  SW = "${TMPDIR}/work-shared/gcc-${PV}-${PR}"
> -SSTATE_SWSPEC = "sstate-gcc-${PV}-${PR}-${SSTATE_VERSION}-"
> +SSTATE_SWSPEC = "sstate:gcc::${PV}:${PR}::${SSTATE_VERSION}:"
>  WORKDIR_task-unpack = "${SW}"
>  WORKDIR_task-patch = "${SW}"
>  
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Richard Purdie - Dec. 20, 2013, 12:37 p.m.
On Fri, 2013-12-20 at 13:33 +0100, Martin Jansa wrote:
> On Fri, Dec 20, 2013 at 12:06:02PM +0000, Richard Purdie wrote:
> > Currently the code has problems differentiating between "gcc-cross" and "gcc-cross-initial"
> > sstate files. We could add in a ton of special casing but tests show this isn't scaling
> > well. Using a more unique separator resolves the issue.
> > 
> > The choice of which separator to use is a hard one. We need something which isn't commonly
> > used in PN, PV, PR, *_OS and *_ARCH which rules out '-', '_' and it needs to work ok with
> > webservers/http which makes ';' and '%' harder.
> > 
> > The change also sets SSTATE_SWSPEC globally since writing out differently named siginfo
> > files for the fetch/unpack/patch tasks is a waste of diskspace, the hashes match for
> > all PN in the majority of cases and if they don't, its not a big issue as the hash is
> > different. This makes the results from sstate debugging more understandable.
> 
> Can you mention why :: is sometimes used in SWSPEC?
> 
> I guess it's again just for easier differentianting, but maybe it should
> be mentioned in commit message.

Its so there are the same number of fields in there, its just empty for
the non-arch specific cases. 

Cheers,

Richard

Patch

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 199a5b3..324d1c1 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -9,8 +9,8 @@  def generate_sstatefn(spec, hash, d):
     return hash[:2] + "/" + spec + hash
 
 SSTATE_PKGARCH    = "${PACKAGE_ARCH}"
-SSTATE_PKGSPEC    = "sstate-${PN}-${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}-${PV}-${PR}-${SSTATE_PKGARCH}-${SSTATE_VERSION}-"
-SSTATE_SWSPEC     = ""
+SSTATE_PKGSPEC    = "sstate:${PN}:${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}:${PV}:${PR}:${SSTATE_PKGARCH}:${SSTATE_VERSION}:"
+SSTATE_SWSPEC     = "sstate:${BPN}::${PV}:${PR}::${SSTATE_VERSION}:"
 SSTATE_PKGNAME    = "${SSTATE_EXTRAPATH}${@generate_sstatefn(d.getVar('SSTATE_PKGSPEC', True), d.getVar('BB_TASKHASH', True), d)}"
 SSTATE_PKG        = "${SSTATE_DIR}/${SSTATE_PKGNAME}"
 SSTATE_EXTRAPATH   = ""
diff --git a/meta/recipes-devtools/gcc/gcc-common.inc b/meta/recipes-devtools/gcc/gcc-common.inc
index e49d724..112e8ce 100644
--- a/meta/recipes-devtools/gcc/gcc-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-common.inc
@@ -95,7 +95,7 @@  do_patch[stamp-base-clean] = "${SSCLEAN}"
 
 # SW means Shared Work directory
 SW = "${TMPDIR}/work-shared/gcc-${PV}-${PR}"
-SSTATE_SWSPEC = "sstate-gcc-${PV}-${PR}-${SSTATE_VERSION}-"
+SSTATE_SWSPEC = "sstate:gcc::${PV}:${PR}::${SSTATE_VERSION}:"
 WORKDIR_task-unpack = "${SW}"
 WORKDIR_task-patch = "${SW}"