Patchwork [0/5] Remove /var/cache from volatiles directory

login
register
mail settings
Submitter Laurentiu Palcu
Date Feb. 4, 2013, 9:26 a.m.
Message ID <cover.1359968730.git.laurentiu.palcu@intel.com>
Download mbox
Permalink /patch/43953/
State New
Headers show

Pull-request

git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles

Comments

Laurentiu Palcu - Feb. 4, 2013, 9:26 a.m.
Hi,

Re-generating applications' cache every time the system is rebooted is not a
very efficient process for an embedded device. Usually, the cache directory is
used by applications to store data resulting from time consmuming I/O or
calculation. So, this patchset will take out /var/cache from tmpfs.

Thanks,
Laurentiu

The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:

  qt4: Add space for _appends (2013-02-01 22:49:47 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles

Laurentiu Palcu (5):
  base-files: remove /var/cache from volatiles
  fs-perms: remove /var/cache from volatiles
  initscripts: remove /var/cache from volatiles
  systemd: remove /var/cache from volatiles
  rpm: remove /var/volatiles/cache/rpm from the FILEs list

 meta/files/fs-perms.txt                            |    1 -
 meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
 .../initscripts/initscripts-1.0/volatiles          |    2 --
 .../systemd/systemd/00-create-volatile.conf        |    1 -
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
 5 files changed, 2 insertions(+), 7 deletions(-)
mike.looijmans@topic.nl - Feb. 4, 2013, 9:46 a.m.
On 02/04/2013 10:26 AM, Laurentiu Palcu wrote:
> Hi,
>
> Re-generating applications' cache every time the system is rebooted is not a
> very efficient process for an embedded device. Usually, the cache directory is
> used by applications to store data resulting from time consmuming I/O or
> calculation. So, this patchset will take out /var/cache from tmpfs.

What if you have a READONLY rootfs? This patch will "break" those 
devices I think.


>
> Thanks,
> Laurentiu
>
> The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:
>
>    qt4: Add space for _appends (2013-02-01 22:49:47 +0000)
>
> are available in the git repository at:
>
>    git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
>    http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles
>
> Laurentiu Palcu (5):
>    base-files: remove /var/cache from volatiles
>    fs-perms: remove /var/cache from volatiles
>    initscripts: remove /var/cache from volatiles
>    systemd: remove /var/cache from volatiles
>    rpm: remove /var/volatiles/cache/rpm from the FILEs list
>
>   meta/files/fs-perms.txt                            |    1 -
>   meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
>   .../initscripts/initscripts-1.0/volatiles          |    2 --
>   .../systemd/systemd/00-create-volatile.conf        |    1 -
>   meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
>   5 files changed, 2 insertions(+), 7 deletions(-)
>
Laurentiu Palcu - Feb. 4, 2013, 10:03 a.m.
On 02/04/2013 11:46 AM, Mike Looijmans wrote:
> On 02/04/2013 10:26 AM, Laurentiu Palcu wrote:
>> > Hi,
>> >
>> > Re-generating applications' cache every time the system is rebooted is not a
>> > very efficient process for an embedded device. Usually, the cache directory is
>> > used by applications to store data resulting from time consmuming I/O or
>> > calculation. So, this patchset will take out /var/cache from tmpfs.
> What if you have a READONLY rootfs? This patch will "break" those 
> devices I think.
> 
> 

Not necessarily. We are trying to have all postinstalls run on host, so
we don't have to do any postinstall activity on target. Any cache
generated during postinstall, will be generated at do_rootfs time. Then,
we can deploy the image with the cache already in place.

Also, on a RO rootfs, there is really no need to update the cache since
the cached files do not really change. Hence, the cache created on host
should, in theory, be valid.

If the above logic is missing anything, feel free to correct me.

Thanks,
Laurentiu
Richard Purdie - Feb. 4, 2013, 11:51 a.m.
On Mon, 2013-02-04 at 12:03 +0200, Laurentiu Palcu wrote:
> 
> On 02/04/2013 11:46 AM, Mike Looijmans wrote:
> > On 02/04/2013 10:26 AM, Laurentiu Palcu wrote:
> >> > Hi,
> >> >
> >> > Re-generating applications' cache every time the system is rebooted is not a
> >> > very efficient process for an embedded device. Usually, the cache directory is
> >> > used by applications to store data resulting from time consmuming I/O or
> >> > calculation. So, this patchset will take out /var/cache from tmpfs.
> > What if you have a READONLY rootfs? This patch will "break" those 
> > devices I think.
> > 
> > 
> 
> Not necessarily. We are trying to have all postinstalls run on host, so
> we don't have to do any postinstall activity on target. Any cache
> generated during postinstall, will be generated at do_rootfs time. Then,
> we can deploy the image with the cache already in place.
> 
> Also, on a RO rootfs, there is really no need to update the cache since
> the cached files do not really change. Hence, the cache created on host
> should, in theory, be valid.
> 
> If the above logic is missing anything, feel free to correct me.

If this is an issue and something does need to touch the cache
directory, we can mount a unionfs over /var/cache like we're doing
for /var/lib in the readonly rootfs patch series.

Ultimately, I think we're better served with that approach all round.

Cheers,

Richard
Phil Blundell - Feb. 4, 2013, 5:47 p.m.
On Mon, 2013-02-04 at 12:03 +0200, Laurentiu Palcu wrote:
> Not necessarily. We are trying to have all postinstalls run on host, so
> we don't have to do any postinstall activity on target. Any cache
> generated during postinstall, will be generated at do_rootfs time. Then,
> we can deploy the image with the cache already in place.
> 
> Also, on a RO rootfs, there is really no need to update the cache since
> the cached files do not really change. Hence, the cache created on host
> should, in theory, be valid.

What sort of files in /var/cache are you trying to address with this
change?  The impression I have is that most of the things in there are
created dynamically by applications running on the target system and not
by postinst scripts that could be run on the host.

For example, an (admittedly somewhat unscientific) survey on my desktop
machine suggests that fontconfig, samba and cups, at least, are creating
files in /var/cache at run time with contents that you wouldn't be able
to predict in advance.  (Apt also uses /var/cache as its download area,
but I guess if you have a read-only roofs then you aren't going to be
running apt very much so that one probably isn't an issue.)  I wasn't
able to immediately identify any files in there that would be obvious
candidates for creating in a postinst and then leaving untouched
afterwards.

p.
Ross Burton - Feb. 4, 2013, 5:51 p.m.
On 4 February 2013 17:47, Phil Blundell <pb@pbcl.net> wrote:
> For example, an (admittedly somewhat unscientific) survey on my desktop
> machine suggests that fontconfig, samba and cups, at least, are creating
> files in /var/cache at run time with contents that you wouldn't be able
> to predict in advance

The fontconfig cache is perfectly capable of generation at image
creation time, and that's actually what prompted removal of /var/cache
from volatiles.

If samba and cups are using /var/cache as something that isn't a
persistent cache, then the should use something like /var/run/
instead.

Ross
Enrico Scholz - Feb. 4, 2013, 6:25 p.m.
"Burton, Ross" <ross.burton-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
writes:

> The fontconfig cache is perfectly capable of generation at image
> creation time,

really? The cache files are architecture dependent and I am not aware of
a cross-fc-cache tool.



Enrico
Ross Burton - Feb. 4, 2013, 7:36 p.m.
On 4 February 2013 18:25, Enrico Scholz <enrico.scholz@sigma-chemnitz.de> wrote:
> really? The cache files are architecture dependent and I am not aware of
> a cross-fc-cache tool.

That's right, they are.  Laurentiu's been working on that too:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=lpalcu/fontconfig&id=fe7c0e0b2c92c36466ac6de02a98d13b3985282e

Ross
Qi.Chen@windriver.com - Feb. 5, 2013, 2:37 a.m.
On 02/04/2013 05:26 PM, Laurentiu Palcu wrote:
> Hi,
>
> Re-generating applications' cache every time the system is rebooted is not a
> very efficient process for an embedded device. Usually, the cache directory is
> used by applications to store data resulting from time consmuming I/O or
> calculation. So, this patchset will take out /var/cache from tmpfs.
>
> Thanks,
> Laurentiu
>
> The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:
>
>    qt4: Add space for _appends (2013-02-01 22:49:47 +0000)
>
> are available in the git repository at:
>
>    git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
>    http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles
>
> Laurentiu Palcu (5):
>    base-files: remove /var/cache from volatiles
>    fs-perms: remove /var/cache from volatiles
>    initscripts: remove /var/cache from volatiles
>    systemd: remove /var/cache from volatiles
>    rpm: remove /var/volatiles/cache/rpm from the FILEs list
>
>   meta/files/fs-perms.txt                            |    1 -
>   meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
>   .../initscripts/initscripts-1.0/volatiles          |    2 --
>   .../systemd/systemd/00-create-volatile.conf        |    1 -
>   meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
>   5 files changed, 2 insertions(+), 7 deletions(-)
>
what about /var/log?

https://bugzilla.yoctoproject.org/show_bug.cgi?id=3397
Laurentiu Palcu - Feb. 5, 2013, 9:56 a.m.
On 02/05/2013 04:37 AM, ChenQi wrote:
> what about /var/log?
> 
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=3397
I wasn't aware this bug existed... Sorry for the inconvenience. I'm not
working on moving /var/log. I had to move the /var/cache though because
of the postinstall work I've been doing.

Thanks,
Laurentiu
Saul Wold - Feb. 5, 2013, 8:50 p.m.
On 02/04/2013 01:26 AM, Laurentiu Palcu wrote:
> Hi,
>
> Re-generating applications' cache every time the system is rebooted is not a
> very efficient process for an embedded device. Usually, the cache directory is
> used by applications to store data resulting from time consmuming I/O or
> calculation. So, this patchset will take out /var/cache from tmpfs.
>
I did a basic test with this change along with the "read-only-rootfs" 
IMAGE_FEATURE and things are looking pretty good.  The only thing I 
noticed is that we leave some /etc/*- files around, I know it's not this 
patch.

Acked-by: Saul Wold <sgw@linux.intel.com>

Thanks
	Sau!
> Thanks,
> Laurentiu
>
> The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:
>
>    qt4: Add space for _appends (2013-02-01 22:49:47 +0000)
>
> are available in the git repository at:
>
>    git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
>    http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles
>
> Laurentiu Palcu (5):
>    base-files: remove /var/cache from volatiles
>    fs-perms: remove /var/cache from volatiles
>    initscripts: remove /var/cache from volatiles
>    systemd: remove /var/cache from volatiles
>    rpm: remove /var/volatiles/cache/rpm from the FILEs list
>
>   meta/files/fs-perms.txt                            |    1 -
>   meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
>   .../initscripts/initscripts-1.0/volatiles          |    2 --
>   .../systemd/systemd/00-create-volatile.conf        |    1 -
>   meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
>   5 files changed, 2 insertions(+), 7 deletions(-)
>