Patchwork [RFC,0/3] Shared state for all !

login
register
mail settings
Submitter Joshua Lock
Date May 10, 2012, 12:22 a.m.
Message ID <cover.1336608479.git.josh@linux.intel.com>
Download mbox
Permalink /patch/27423/
State New
Headers show

Pull-request

git://git.yoctoproject.org/poky-contrib josh/perms

Comments

Joshua Lock - May 10, 2012, 12:22 a.m.
This is an RFC series, and for ease of review sent as against the Poky tree so
that it can be reviewed as a whole.

In Yocto #2041[2]  Mark reported an issue with reusing shared state as a
different user on the same machine.

Since the whole purpose of shared state is that it be shared I decided to dig
into this issue. I wanted to at least be able to use the shared-state cache of
a different user without error, even if all of the objects aren't actually used
(i.e. native, at least on the Edison branch I did most of the testing with).

This is an RFC mainly because it changes the permissions of created directories,
sstate files and siginfo files from what they have traditionally been.

There is more of the rhyme an reason in the patch commit headers and comments
but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this patch, as
will all of the contents of sstate-cache (siginfo and tgz) files.

This is actually what one would expect from reading the Python API docs for
os.makedirs "The default mode is 0777 (octal)."[1] but not what actually happens
on most modern Linux systems thanks to umask.

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

Regards,

Joshua

1. http://docs.python.org/library/os.html#os.makedirs
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=2041

The following changes since commit 6841abd17268a1c1ca5e3634f0ae9e21426ca9c9:

  Hob: change the workflow about click 'Run Image' to run directly after builded a qemu image (2012-05-09 21:56:47 +0100)

are available in the git repository at:
  git://git.yoctoproject.org/poky-contrib josh/perms
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=josh/perms

just the BitBake changes are at:
  git://github.com/incandescant/bitbake.git josh/perms
  https://github.com/incandescant/bitbake/commits/josh/perms

just the openembedded-core changes are at:
  git://git.openembedded.org/openembedded-core-contrib josh/perms
  http://git.openembedded.org/openembedded-core-contrib/log/?h=josh/perms

Joshua Lock (3):
  lib/bb/utils.py: add optional mode parameter to bb.utils.mkdirhier()
  lib/bb/siggen.py: create permissive files and directories
  sstate.bbclass: ensure sstate files are easily shared

 bitbake/lib/bb/siggen.py    |    7 ++++++-
 bitbake/lib/bb/utils.py     |    7 +++++--
 meta/classes/sstate.bbclass |    1 +
 3 files changed, 12 insertions(+), 3 deletions(-)
Chris Larson - May 10, 2012, 12:50 a.m.
On Wed, May 9, 2012 at 5:22 PM, Joshua Lock <josh@linux.intel.com> wrote:
> In Yocto #2041[2]  Mark reported an issue with reusing shared state as a
> different user on the same machine.
>
> Since the whole purpose of shared state is that it be shared I decided to dig
> into this issue. I wanted to at least be able to use the shared-state cache of
> a different user without error, even if all of the objects aren't actually used
> (i.e. native, at least on the Edison branch I did most of the testing with).
>
> This is an RFC mainly because it changes the permissions of created directories,
> sstate files and siginfo files from what they have traditionally been.
>
> There is more of the rhyme an reason in the patch commit headers and comments
> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this patch, as
> will all of the contents of sstate-cache (siginfo and tgz) files.
>
> This is actually what one would expect from reading the Python API docs for
> os.makedirs "The default mode is 0777 (octal)."[1] but not what actually happens
> on most modern Linux systems thanks to umask.
>
> Please review the following changes for suitability for inclusion. If you have
> any objections or suggestions for improvement, please respond to the patches. If
> you agree with the changes, please provide your Acked-by.

777 seems questionable to me, personally. Generally collaboration
happens amongst folks within a group, and chmod g+s makes that easier.
I'd expect 775 to be a more sane value, myself.
Joshua Lock - May 10, 2012, 1:05 a.m.
On 09/05/12 17:50, Chris Larson wrote:
> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>  wrote:
>> In Yocto #2041[2]  Mark reported an issue with reusing shared state as a
>> different user on the same machine.
>>
>> Since the whole purpose of shared state is that it be shared I decided to dig
>> into this issue. I wanted to at least be able to use the shared-state cache of
>> a different user without error, even if all of the objects aren't actually used
>> (i.e. native, at least on the Edison branch I did most of the testing with).
>>
>> This is an RFC mainly because it changes the permissions of created directories,
>> sstate files and siginfo files from what they have traditionally been.
>>
>> There is more of the rhyme an reason in the patch commit headers and comments
>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this patch, as
>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>
>> This is actually what one would expect from reading the Python API docs for
>> os.makedirs "The default mode is 0777 (octal)."[1] but not what actually happens
>> on most modern Linux systems thanks to umask.
>>
>> Please review the following changes for suitability for inclusion. If you have
>> any objections or suggestions for improvement, please respond to the patches. If
>> you agree with the changes, please provide your Acked-by.
>
> 777 seems questionable to me, personally. Generally collaboration
> happens amongst folks within a group, and chmod g+s makes that easier.
> I'd expect 775 to be a more sane value, myself.

Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or 
everything?

I went with 777 for mkdirhier as that's the default of os.makedirs 
before umask is involved. I would likely have picked rw-rw-r-- (664) if 
I weren't trying to request comments.

Cheers,
Joshua
Chris Larson - May 10, 2012, 2:15 a.m.
On Wed, May 9, 2012 at 6:05 PM, Joshua Lock <josh@linux.intel.com> wrote:
>
>
> On 09/05/12 17:50, Chris Larson wrote:
>>
>> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>  wrote:
>>>
>>> In Yocto #2041[2]  Mark reported an issue with reusing shared state as a
>>> different user on the same machine.
>>>
>>> Since the whole purpose of shared state is that it be shared I decided to
>>> dig
>>> into this issue. I wanted to at least be able to use the shared-state
>>> cache of
>>> a different user without error, even if all of the objects aren't
>>> actually used
>>> (i.e. native, at least on the Edison branch I did most of the testing
>>> with).
>>>
>>> This is an RFC mainly because it changes the permissions of created
>>> directories,
>>> sstate files and siginfo files from what they have traditionally been.
>>>
>>> There is more of the rhyme an reason in the patch commit headers and
>>> comments
>>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this
>>> patch, as
>>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>>
>>> This is actually what one would expect from reading the Python API docs
>>> for
>>> os.makedirs "The default mode is 0777 (octal)."[1] but not what actually
>>> happens
>>> on most modern Linux systems thanks to umask.
>>>
>>> Please review the following changes for suitability for inclusion. If you
>>> have
>>> any objections or suggestions for improvement, please respond to the
>>> patches. If
>>> you agree with the changes, please provide your Acked-by.
>>
>>
>> 777 seems questionable to me, personally. Generally collaboration
>> happens amongst folks within a group, and chmod g+s makes that easier.
>> I'd expect 775 to be a more sane value, myself.
>
>
> Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or
> everything?
>
> I went with 777 for mkdirhier as that's the default of os.makedirs before
> umask is involved. I would likely have picked rw-rw-r-- (664) if I weren't
> trying to request comments.

Gotcha.

I'm concerned about the behavior change and potential implications of
changing the default behavior of mkdirhier. I'm inclined to say that
when you don't pass mode, let it use the current behavior of obeying
the umask. If we're not going to do that, and want to change the
default behavior, then I think 777 is the wrong/questionable default.
Beyond that, 777 is certainly the wrong mode to be using for the
shared state package in sstate.bbclass.
Joshua Lock - May 10, 2012, 2:32 a.m.
On 09/05/12 19:15, Chris Larson wrote:
> On Wed, May 9, 2012 at 6:05 PM, Joshua Lock<josh@linux.intel.com>  wrote:
>>
>>
>> On 09/05/12 17:50, Chris Larson wrote:
>>>
>>> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>    wrote:
>>>>
>>>> In Yocto #2041[2]  Mark reported an issue with reusing shared state as a
>>>> different user on the same machine.
>>>>
>>>> Since the whole purpose of shared state is that it be shared I decided to
>>>> dig
>>>> into this issue. I wanted to at least be able to use the shared-state
>>>> cache of
>>>> a different user without error, even if all of the objects aren't
>>>> actually used
>>>> (i.e. native, at least on the Edison branch I did most of the testing
>>>> with).
>>>>
>>>> This is an RFC mainly because it changes the permissions of created
>>>> directories,
>>>> sstate files and siginfo files from what they have traditionally been.
>>>>
>>>> There is more of the rhyme an reason in the patch commit headers and
>>>> comments
>>>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this
>>>> patch, as
>>>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>>>
>>>> This is actually what one would expect from reading the Python API docs
>>>> for
>>>> os.makedirs "The default mode is 0777 (octal)."[1] but not what actually
>>>> happens
>>>> on most modern Linux systems thanks to umask.
>>>>
>>>> Please review the following changes for suitability for inclusion. If you
>>>> have
>>>> any objections or suggestions for improvement, please respond to the
>>>> patches. If
>>>> you agree with the changes, please provide your Acked-by.
>>>
>>>
>>> 777 seems questionable to me, personally. Generally collaboration
>>> happens amongst folks within a group, and chmod g+s makes that easier.
>>> I'd expect 775 to be a more sane value, myself.
>>
>>
>> Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or
>> everything?
>>
>> I went with 777 for mkdirhier as that's the default of os.makedirs before
>> umask is involved. I would likely have picked rw-rw-r-- (664) if I weren't
>> trying to request comments.
>
> Gotcha.
>
> I'm concerned about the behavior change and potential implications of
> changing the default behavior of mkdirhier. I'm inclined to say that
> when you don't pass mode, let it use the current behavior of obeying
> the umask.

An earlier version of the series did this and I'm happy to add that 
behaviour back in.

  If we're not going to do that, and want to change the
> default behavior, then I think 777 is the wrong/questionable default.
> Beyond that, 777 is certainly the wrong mode to be using for the
> shared state package in sstate.bbclass.

Do you have a strong preference on 664 vs. 775 ?

Thanks for the comments and feedback,

Joshua
Joshua Lock - May 10, 2012, 3:10 a.m.
On 09/05/12 19:32, Joshua Lock wrote:
> On 09/05/12 19:15, Chris Larson wrote:
>> On Wed, May 9, 2012 at 6:05 PM, Joshua Lock<josh@linux.intel.com> wrote:
>>>
>>>
>>> On 09/05/12 17:50, Chris Larson wrote:
>>>>
>>>> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>
>>>> wrote:
>>>>>
>>>>> In Yocto #2041[2] Mark reported an issue with reusing shared state
>>>>> as a
>>>>> different user on the same machine.
>>>>>
>>>>> Since the whole purpose of shared state is that it be shared I
>>>>> decided to
>>>>> dig
>>>>> into this issue. I wanted to at least be able to use the shared-state
>>>>> cache of
>>>>> a different user without error, even if all of the objects aren't
>>>>> actually used
>>>>> (i.e. native, at least on the Edison branch I did most of the testing
>>>>> with).
>>>>>
>>>>> This is an RFC mainly because it changes the permissions of created
>>>>> directories,
>>>>> sstate files and siginfo files from what they have traditionally been.
>>>>>
>>>>> There is more of the rhyme an reason in the patch commit headers and
>>>>> comments
>>>>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this
>>>>> patch, as
>>>>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>>>>
>>>>> This is actually what one would expect from reading the Python API
>>>>> docs
>>>>> for
>>>>> os.makedirs "The default mode is 0777 (octal)."[1] but not what
>>>>> actually
>>>>> happens
>>>>> on most modern Linux systems thanks to umask.
>>>>>
>>>>> Please review the following changes for suitability for inclusion.
>>>>> If you
>>>>> have
>>>>> any objections or suggestions for improvement, please respond to the
>>>>> patches. If
>>>>> you agree with the changes, please provide your Acked-by.
>>>>
>>>>
>>>> 777 seems questionable to me, personally. Generally collaboration
>>>> happens amongst folks within a group, and chmod g+s makes that easier.
>>>> I'd expect 775 to be a more sane value, myself.
>>>
>>>
>>> Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or
>>> everything?
>>>
>>> I went with 777 for mkdirhier as that's the default of os.makedirs
>>> before
>>> umask is involved. I would likely have picked rw-rw-r-- (664) if I
>>> weren't
>>> trying to request comments.
>>
>> Gotcha.
>>
>> I'm concerned about the behavior change and potential implications of
>> changing the default behavior of mkdirhier. I'm inclined to say that
>> when you don't pass mode, let it use the current behavior of obeying
>> the umask.
>
> An earlier version of the series did this and I'm happy to add that
> behaviour back in.
>
> If we're not going to do that, and want to change the
>> default behavior, then I think 777 is the wrong/questionable default.
>> Beyond that, 777 is certainly the wrong mode to be using for the
>> shared state package in sstate.bbclass.
>
> Do you have a strong preference on 664 vs. 775 ?

I'm leaning towards 664 (rw-rw-r--) for the files and 775 (rwxrwxr-x) 
for directories, these are the defaults for file and directory creation 
on Ubuntu 12.04 and Fedora 16.

Cheers,
Joshua
Joshua Lock - May 10, 2012, 3:14 a.m.
On 09/05/12 20:10, Joshua Lock wrote:
> On 09/05/12 19:32, Joshua Lock wrote:
>> On 09/05/12 19:15, Chris Larson wrote:
>>> On Wed, May 9, 2012 at 6:05 PM, Joshua Lock<josh@linux.intel.com> wrote:
>>>>
>>>>
>>>> On 09/05/12 17:50, Chris Larson wrote:
>>>>>
>>>>> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>
>>>>> wrote:
>>>>>>
>>>>>> In Yocto #2041[2] Mark reported an issue with reusing shared state
>>>>>> as a
>>>>>> different user on the same machine.
>>>>>>
>>>>>> Since the whole purpose of shared state is that it be shared I
>>>>>> decided to
>>>>>> dig
>>>>>> into this issue. I wanted to at least be able to use the shared-state
>>>>>> cache of
>>>>>> a different user without error, even if all of the objects aren't
>>>>>> actually used
>>>>>> (i.e. native, at least on the Edison branch I did most of the testing
>>>>>> with).
>>>>>>
>>>>>> This is an RFC mainly because it changes the permissions of created
>>>>>> directories,
>>>>>> sstate files and siginfo files from what they have traditionally
>>>>>> been.
>>>>>>
>>>>>> There is more of the rhyme an reason in the patch commit headers and
>>>>>> comments
>>>>>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this
>>>>>> patch, as
>>>>>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>>>>>
>>>>>> This is actually what one would expect from reading the Python API
>>>>>> docs
>>>>>> for
>>>>>> os.makedirs "The default mode is 0777 (octal)."[1] but not what
>>>>>> actually
>>>>>> happens
>>>>>> on most modern Linux systems thanks to umask.
>>>>>>
>>>>>> Please review the following changes for suitability for inclusion.
>>>>>> If you
>>>>>> have
>>>>>> any objections or suggestions for improvement, please respond to the
>>>>>> patches. If
>>>>>> you agree with the changes, please provide your Acked-by.
>>>>>
>>>>>
>>>>> 777 seems questionable to me, personally. Generally collaboration
>>>>> happens amongst folks within a group, and chmod g+s makes that easier.
>>>>> I'd expect 775 to be a more sane value, myself.
>>>>
>>>>
>>>> Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or
>>>> everything?
>>>>
>>>> I went with 777 for mkdirhier as that's the default of os.makedirs
>>>> before
>>>> umask is involved. I would likely have picked rw-rw-r-- (664) if I
>>>> weren't
>>>> trying to request comments.
>>>
>>> Gotcha.
>>>
>>> I'm concerned about the behavior change and potential implications of
>>> changing the default behavior of mkdirhier. I'm inclined to say that
>>> when you don't pass mode, let it use the current behavior of obeying
>>> the umask.
>>
>> An earlier version of the series did this and I'm happy to add that
>> behaviour back in.
>>
>> If we're not going to do that, and want to change the
>>> default behavior, then I think 777 is the wrong/questionable default.
>>> Beyond that, 777 is certainly the wrong mode to be using for the
>>> shared state package in sstate.bbclass.
>>
>> Do you have a strong preference on 664 vs. 775 ?
>
> I'm leaning towards 664 (rw-rw-r--) for the files and 775 (rwxrwxr-x)
> for directories, these are the defaults for file and directory creation
> on Ubuntu 12.04 and Fedora 16.

At which point, the obvious "solution" to the bug report is a sanity 
check whether the user can read and write to the sstate directory.

Cheers,
Joshua
Khem Raj - May 10, 2012, 5:16 a.m.
On Wed, May 9, 2012 at 8:14 PM, Joshua Lock <josh@linux.intel.com> wrote:
> On 09/05/12 20:10, Joshua Lock wrote:
>>
>> On 09/05/12 19:32, Joshua Lock wrote:
>>>
>>> On 09/05/12 19:15, Chris Larson wrote:
>>>>
>>>> On Wed, May 9, 2012 at 6:05 PM, Joshua Lock<josh@linux.intel.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 09/05/12 17:50, Chris Larson wrote:
>>>>>>
>>>>>>
>>>>>> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> In Yocto #2041[2] Mark reported an issue with reusing shared state
>>>>>>> as a
>>>>>>> different user on the same machine.
>>>>>>>
>>>>>>> Since the whole purpose of shared state is that it be shared I
>>>>>>> decided to
>>>>>>> dig
>>>>>>> into this issue. I wanted to at least be able to use the shared-state
>>>>>>> cache of
>>>>>>> a different user without error, even if all of the objects aren't
>>>>>>> actually used
>>>>>>> (i.e. native, at least on the Edison branch I did most of the testing
>>>>>>> with).
>>>>>>>
>>>>>>> This is an RFC mainly because it changes the permissions of created
>>>>>>> directories,
>>>>>>> sstate files and siginfo files from what they have traditionally
>>>>>>> been.
>>>>>>>
>>>>>>> There is more of the rhyme an reason in the patch commit headers and
>>>>>>> comments
>>>>>>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this
>>>>>>> patch, as
>>>>>>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>>>>>>
>>>>>>> This is actually what one would expect from reading the Python API
>>>>>>> docs
>>>>>>> for
>>>>>>> os.makedirs "The default mode is 0777 (octal)."[1] but not what
>>>>>>> actually
>>>>>>> happens
>>>>>>> on most modern Linux systems thanks to umask.
>>>>>>>
>>>>>>> Please review the following changes for suitability for inclusion.
>>>>>>> If you
>>>>>>> have
>>>>>>> any objections or suggestions for improvement, please respond to the
>>>>>>> patches. If
>>>>>>> you agree with the changes, please provide your Acked-by.
>>>>>>
>>>>>>
>>>>>>
>>>>>> 777 seems questionable to me, personally. Generally collaboration
>>>>>> happens amongst folks within a group, and chmod g+s makes that easier.
>>>>>> I'd expect 775 to be a more sane value, myself.
>>>>>
>>>>>
>>>>>
>>>>> Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or
>>>>> everything?
>>>>>
>>>>> I went with 777 for mkdirhier as that's the default of os.makedirs
>>>>> before
>>>>> umask is involved. I would likely have picked rw-rw-r-- (664) if I
>>>>> weren't
>>>>> trying to request comments.
>>>>
>>>>
>>>> Gotcha.
>>>>
>>>> I'm concerned about the behavior change and potential implications of
>>>> changing the default behavior of mkdirhier. I'm inclined to say that
>>>> when you don't pass mode, let it use the current behavior of obeying
>>>> the umask.
>>>
>>>
>>> An earlier version of the series did this and I'm happy to add that
>>> behaviour back in.
>>>
>>> If we're not going to do that, and want to change the
>>>>
>>>> default behavior, then I think 777 is the wrong/questionable default.
>>>> Beyond that, 777 is certainly the wrong mode to be using for the
>>>> shared state package in sstate.bbclass.
>>>
>>>
>>> Do you have a strong preference on 664 vs. 775 ?
>>
>>
>> I'm leaning towards 664 (rw-rw-r--) for the files and 775 (rwxrwxr-x)
>> for directories, these are the defaults for file and directory creation
>> on Ubuntu 12.04 and Fedora 16.
>
>
> At which point, the obvious "solution" to the bug report is a sanity check
> whether the user can read and write to the sstate directory.

may be its already taken care of but I will ask anyway, if shared
state is writable for all
how is contention resolved when someone tries to update the shared
state since he/she rebuilt the package for some reason. I think having
a global shared cache to read from but not update it would be
something interesting where shared state is updated by say an
autobuilder and used by developers in their local builds.

On perm issue  could it be made that if the file is readable by user
then while unpacking it gets the umask of current user ?

-Khem
Koen Kooi - May 10, 2012, 7:25 a.m.
Op 10 mei 2012, om 02:22 heeft Joshua Lock het volgende geschreven:

> This is an RFC series, and for ease of review sent as against the Poky tree

It requires an extra step to apply and thus makes review harder for me since my motivation to test them drops below zero. Please send patches to bitbake and oe-core to this list and reserve poky patches for the poky list.
As for ease of review by glancing over them: 3 emails are 3 emails.
Joshua Lock - May 10, 2012, 4:05 p.m.
On 10/05/12 00:25, Koen Kooi wrote:
>
> Op 10 mei 2012, om 02:22 heeft Joshua Lock het volgende geschreven:
>
>> This is an RFC series, and for ease of review sent as against the Poky tree
>
> It requires an extra step to apply and thus makes review harder for me since my motivation to test them drops below zero. Please send patches to bitbake and oe-core to this list and reserve poky patches for the poky list.
> As for ease of review by glancing over them: 3 emails are 3 emails.

BitBake and OE-Core branches were included in the email, though I didn't 
explicitly point that out.

Joshua
Joshua Lock - May 10, 2012, 4:12 p.m.
On 09/05/12 22:16, Khem Raj wrote:
> On Wed, May 9, 2012 at 8:14 PM, Joshua Lock<josh@linux.intel.com>  wrote:
>> On 09/05/12 20:10, Joshua Lock wrote:
>>>
>>> On 09/05/12 19:32, Joshua Lock wrote:
>>>>
>>>> On 09/05/12 19:15, Chris Larson wrote:
>>>>>
>>>>> On Wed, May 9, 2012 at 6:05 PM, Joshua Lock<josh@linux.intel.com>  wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 09/05/12 17:50, Chris Larson wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 9, 2012 at 5:22 PM, Joshua Lock<josh@linux.intel.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> In Yocto #2041[2] Mark reported an issue with reusing shared state
>>>>>>>> as a
>>>>>>>> different user on the same machine.
>>>>>>>>
>>>>>>>> Since the whole purpose of shared state is that it be shared I
>>>>>>>> decided to
>>>>>>>> dig
>>>>>>>> into this issue. I wanted to at least be able to use the shared-state
>>>>>>>> cache of
>>>>>>>> a different user without error, even if all of the objects aren't
>>>>>>>> actually used
>>>>>>>> (i.e. native, at least on the Edison branch I did most of the testing
>>>>>>>> with).
>>>>>>>>
>>>>>>>> This is an RFC mainly because it changes the permissions of created
>>>>>>>> directories,
>>>>>>>> sstate files and siginfo files from what they have traditionally
>>>>>>>> been.
>>>>>>>>
>>>>>>>> There is more of the rhyme an reason in the patch commit headers and
>>>>>>>> comments
>>>>>>>> but tl;dr bb.mkdirhier directories will be 0777 (rwxrwxrwx) with this
>>>>>>>> patch, as
>>>>>>>> will all of the contents of sstate-cache (siginfo and tgz) files.
>>>>>>>>
>>>>>>>> This is actually what one would expect from reading the Python API
>>>>>>>> docs
>>>>>>>> for
>>>>>>>> os.makedirs "The default mode is 0777 (octal)."[1] but not what
>>>>>>>> actually
>>>>>>>> happens
>>>>>>>> on most modern Linux systems thanks to umask.
>>>>>>>>
>>>>>>>> Please review the following changes for suitability for inclusion.
>>>>>>>> If you
>>>>>>>> have
>>>>>>>> any objections or suggestions for improvement, please respond to the
>>>>>>>> patches. If
>>>>>>>> you agree with the changes, please provide your Acked-by.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 777 seems questionable to me, personally. Generally collaboration
>>>>>>> happens amongst folks within a group, and chmod g+s makes that easier.
>>>>>>> I'd expect 775 to be a more sane value, myself.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Do you mean for bb.mkdirhier calls, the tgz files, the siginfo files or
>>>>>> everything?
>>>>>>
>>>>>> I went with 777 for mkdirhier as that's the default of os.makedirs
>>>>>> before
>>>>>> umask is involved. I would likely have picked rw-rw-r-- (664) if I
>>>>>> weren't
>>>>>> trying to request comments.
>>>>>
>>>>>
>>>>> Gotcha.
>>>>>
>>>>> I'm concerned about the behavior change and potential implications of
>>>>> changing the default behavior of mkdirhier. I'm inclined to say that
>>>>> when you don't pass mode, let it use the current behavior of obeying
>>>>> the umask.
>>>>
>>>>
>>>> An earlier version of the series did this and I'm happy to add that
>>>> behaviour back in.
>>>>
>>>> If we're not going to do that, and want to change the
>>>>>
>>>>> default behavior, then I think 777 is the wrong/questionable default.
>>>>> Beyond that, 777 is certainly the wrong mode to be using for the
>>>>> shared state package in sstate.bbclass.
>>>>
>>>>
>>>> Do you have a strong preference on 664 vs. 775 ?
>>>
>>>
>>> I'm leaning towards 664 (rw-rw-r--) for the files and 775 (rwxrwxr-x)
>>> for directories, these are the defaults for file and directory creation
>>> on Ubuntu 12.04 and Fedora 16.
>>
>>
>> At which point, the obvious "solution" to the bug report is a sanity check
>> whether the user can read and write to the sstate directory.
>
> may be its already taken care of but I will ask anyway, if shared
> state is writable for all
> how is contention resolved when someone tries to update the shared
> state since he/she rebuilt the package for some reason. I think having
> a global shared cache to read from but not update it would be
> something interesting where shared state is updated by say an
> autobuilder and used by developers in their local builds.

SSTATE_MIRRORS does this nicely and since it uses the fetcher code that 
can be a file:// URI or other.

Joshua
Khem Raj - May 10, 2012, 4:31 p.m.
On Thu, May 10, 2012 at 9:12 AM, Joshua Lock <josh@linux.intel.com> wrote:
>
> SSTATE_MIRRORS does this nicely and since it uses the fetcher code that can
> be a file:// URI or other.

yes figured yesterday night when I was thinking about it loud