Patchwork [01/20] udev-cache: Compress the cache

login
register
mail settings
Submitter Ben Shelton
Date Aug. 4, 2014, 6:40 p.m.
Message ID <78fc1827553729a69ee665bf4ef7e33495d9de0b.1407177403.git.ben.shelton@ni.com>
Download mbox | patch
Permalink /patch/77233/
State Accepted
Commit 8c423bf995dc829c4bbb286e5f1c020fc053e6fc
Headers show

Comments

Ben Shelton - Aug. 4, 2014, 6:40 p.m.
From: Richard Tollerton <rich.tollerton@ni.com>

$DEVCACHE is observed to be 100k uncompressed; compressing it reduces
its size to ~5k.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init               | 2 +-
 meta/recipes-core/udev/udev/udev-cache         | 2 +-
 meta/recipes-core/udev/udev/udev-cache.default | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
Otavio Salvador - Aug. 4, 2014, 7:29 p.m.
On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> its size to ~5k.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>

Acked-by: Otavio Salvador <otavio@ossystems.com.br>
Khem Raj - Aug. 4, 2014, 10:44 p.m.
On 14-08-04 13:40:53, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
> 
> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> its size to ~5k.
> 
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>  meta/recipes-core/udev/udev/init               | 2 +-
>  meta/recipes-core/udev/udev/udev-cache         | 2 +-
>  meta/recipes-core/udev/udev/udev-cache.default | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> index f2c84d5..1e69861 100644
> --- a/meta/recipes-core/udev/udev/init
> +++ b/meta/recipes-core/udev/udev/init
> @@ -69,7 +69,7 @@ case "$1" in
>  		    readfiles /etc/udev/cache.data
>  		    OLDDATA="$READDATA"
>  		    if [ "$OLDDATA" = "$NEWDATA" ]; then
> -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> +                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)

wouldnt cf still handle gz and xz or any other compression ?
Ben Shelton - Aug. 4, 2014, 11:45 p.m.
On 08/04, Khem Raj wrote:
> On 14-08-04 13:40:53, Ben Shelton wrote:
> > From: Richard Tollerton <rich.tollerton@ni.com>
> > 
> > $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> > its size to ~5k.
> > 
> > Natinst-Rally-ID: TA44427
> > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> > Natinst-ReviewBoard-ID: 58620
> > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> > ---
> >  meta/recipes-core/udev/udev/init               | 2 +-
> >  meta/recipes-core/udev/udev/udev-cache         | 2 +-
> >  meta/recipes-core/udev/udev/udev-cache.default | 2 +-
> >  3 files changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> > index f2c84d5..1e69861 100644
> > --- a/meta/recipes-core/udev/udev/init
> > +++ b/meta/recipes-core/udev/udev/init
> > @@ -69,7 +69,7 @@ case "$1" in
> >  		    readfiles /etc/udev/cache.data
> >  		    OLDDATA="$READDATA"
> >  		    if [ "$OLDDATA" = "$NEWDATA" ]; then
> > -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> > +                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
> 
> wouldnt cf still handle gz and xz or any other compression ?

Do you mean xf?  If so, then yes, I think it would.

What would be the advantage of doing it that way?  Just one fewer
change?

Thanks,
Ben
Khem Raj - Aug. 5, 2014, 12:06 a.m.
On Aug 4, 2014 4:45 PM, "Ben Shelton" <ben.shelton@ni.com> wrote:
>
> On 08/04, Khem Raj wrote:
> > On 14-08-04 13:40:53, Ben Shelton wrote:
> > > From: Richard Tollerton <rich.tollerton@ni.com>
> > >
> > > $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> > > its size to ~5k.
> > >
> > > Natinst-Rally-ID: TA44427
> > > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> > > Natinst-ReviewBoard-ID: 58620
> > > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> > > ---
> > >  meta/recipes-core/udev/udev/init               | 2 +-
> > >  meta/recipes-core/udev/udev/udev-cache         | 2 +-
> > >  meta/recipes-core/udev/udev/udev-cache.default | 2 +-
> > >  3 files changed, 3 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/meta/recipes-core/udev/udev/init
b/meta/recipes-core/udev/udev/init
> > > index f2c84d5..1e69861 100644
> > > --- a/meta/recipes-core/udev/udev/init
> > > +++ b/meta/recipes-core/udev/udev/init
> > > @@ -69,7 +69,7 @@ case "$1" in
> > >                 readfiles /etc/udev/cache.data
> > >                 OLDDATA="$READDATA"
> > >                 if [ "$OLDDATA" = "$NEWDATA" ]; then
> > > -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> > > +                            (cd /; tar xzf $DEVCACHE > /dev/null
2>&1)
> >
> > wouldnt cf still handle gz and xz or any other compression ?
>
> Do you mean xf?  If so, then yes, I think it would.
>
> What would be the advantage of doing it that way?  Just one fewer
> change?

yes and tomorrow you might want to use different compression this wont
change
>
> Thanks,
> Ben
>
Randy MacLeod - Aug. 5, 2014, 7 p.m.
On 14-08-04 02:40 PM, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> its size to ~5k.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620


This is a good series of patches; thanks for sending them.

It would be nice if you could remove your company's ID tags
from the commit log. For example, you create and use could use
a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
then match that up in your internal process.

Thanks,
../Randy


> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>   meta/recipes-core/udev/udev/init               | 2 +-
>   meta/recipes-core/udev/udev/udev-cache         | 2 +-
>   meta/recipes-core/udev/udev/udev-cache.default | 2 +-
>   3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> index f2c84d5..1e69861 100644
> --- a/meta/recipes-core/udev/udev/init
> +++ b/meta/recipes-core/udev/udev/init
> @@ -69,7 +69,7 @@ case "$1" in
>   		    readfiles /etc/udev/cache.data
>   		    OLDDATA="$READDATA"
>   		    if [ "$OLDDATA" = "$NEWDATA" ]; then
> -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> +                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
>                               not_first_boot=1
>                               [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
>                               [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
> diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
> index db5a513..3769062 100644
> --- a/meta/recipes-core/udev/udev/udev-cache
> +++ b/meta/recipes-core/udev/udev/udev-cache
> @@ -25,7 +25,7 @@ fi
>
>   if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
>   	echo "Populating dev cache"
> -	(cd /; tar cf "$DEVCACHE" dev)
> +	(cd /; tar czf "$DEVCACHE" dev)
>   	mv /dev/shm/udev.cache /etc/udev/cache.data
>   fi
>
> diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
> index 2093336..909ec87 100644
> --- a/meta/recipes-core/udev/udev/udev-cache.default
> +++ b/meta/recipes-core/udev/udev/udev-cache.default
> @@ -1,5 +1,5 @@
>   # Default for /etc/init.d/udev
>
>   # Comment this out to disable device cache
> -DEVCACHE="/etc/dev.tar"
> +DEVCACHE="/etc/dev.tar.gz"
>   PROBE_PLATFORM_BUS="yes"
>
Otavio Salvador - Aug. 5, 2014, 8:09 p.m.
On Tue, Aug 5, 2014 at 4:00 PM, Randy MacLeod
<randy.macleod@windriver.com> wrote:
> On 14-08-04 02:40 PM, Ben Shelton wrote:
>>
>> From: Richard Tollerton <rich.tollerton@ni.com>
>>
>> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
>> its size to ~5k.
>>
>> Natinst-Rally-ID: TA44427
>> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
>> Natinst-ReviewBoard-ID: 58620
>
>
>
> This is a good series of patches; thanks for sending them.
>
> It would be nice if you could remove your company's ID tags
> from the commit log. For example, you create and use could use
> a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
> then match that up in your internal process.

I don't think we should ask for this. I saw several of us to include
Id tags in past commits (I also did) and I think it is fine. We
shouldn't ask an extra step for people to be able to track it (open a
YP bugzilla ID issue for example).
Randy MacLeod - Aug. 5, 2014, 8:35 p.m.
On 14-08-05 04:09 PM, Otavio Salvador wrote:
> On Tue, Aug 5, 2014 at 4:00 PM, Randy MacLeod
> <randy.macleod@windriver.com> wrote:
>> On 14-08-04 02:40 PM, Ben Shelton wrote:
>>>
>>> From: Richard Tollerton <rich.tollerton@ni.com>
>>>
>>> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
>>> its size to ~5k.
>>>
>>> Natinst-Rally-ID: TA44427
>>> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
>>> Natinst-ReviewBoard-ID: 58620
>>
>>
>>
>> This is a good series of patches; thanks for sending them.
>>
>> It would be nice if you could remove your company's ID tags
>> from the commit log. For example, you create and use could use
>> a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
>> then match that up in your internal process.
>
> I don't think we should ask for this. I saw several of us to include
> Id tags in past commits (I also did) and I think it is fine. We
> shouldn't ask an extra step for people to be able to track it (open a
> YP bugzilla ID issue for example).
>

Wind River patches used to include a "CQID" tag but we've changed
our process to avoid needing such internal tags. If National
Instruments can do so as well, that'd be best.

I did check my oe-core email list history and this seems like the
first patch from NI that has the tags included so I thought
I'd reply and see if we can get the tags dropped.

Ben, what's the story?
Ben Shelton - Aug. 6, 2014, 2:08 a.m.
On 08/05, Randy MacLeod wrote:
> On 14-08-05 04:09 PM, Otavio Salvador wrote:
> >On Tue, Aug 5, 2014 at 4:00 PM, Randy MacLeod
> ><randy.macleod@windriver.com> wrote:
> >>On 14-08-04 02:40 PM, Ben Shelton wrote:
> >>>
> >>>From: Richard Tollerton <rich.tollerton@ni.com>
> >>>
> >>>$DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> >>>its size to ~5k.
> >>>
> >>>Natinst-Rally-ID: TA44427
> >>>Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> >>>Natinst-ReviewBoard-ID: 58620
> >>
> >>
> >>
> >>This is a good series of patches; thanks for sending them.
> >>
> >>It would be nice if you could remove your company's ID tags
> >>from the commit log. For example, you create and use could use
> >>a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
> >>then match that up in your internal process.
> >
> >I don't think we should ask for this. I saw several of us to include
> >Id tags in past commits (I also did) and I think it is fine. We
> >shouldn't ask an extra step for people to be able to track it (open a
> >YP bugzilla ID issue for example).
> >
> 
> Wind River patches used to include a "CQID" tag but we've changed
> our process to avoid needing such internal tags. If National
> Instruments can do so as well, that'd be best.
> 
> I did check my oe-core email list history and this seems like the
> first patch from NI that has the tags included so I thought
> I'd reply and see if we can get the tags dropped.
> 
> Ben, what's the story?

In the past, I've stripped the tags before posting the patches.  Looks
like I forgot this time -- I'll remove the tags when I submit v2 of the
patch set.

Thanks,
Ben

> 
> -- 
> # Randy MacLeod. SMTS, Linux, Wind River
> Direct: 613.963.1350
Richard Tollerton - Aug. 22, 2014, 3:24 p.m.
Randy MacLeod <randy.macleod@windriver.com> writes:

> Wind River patches used to include a "CQID" tag but we've changed
> our process to avoid needing such internal tags. If National
> Instruments can do so as well, that'd be best.
>
> I did check my oe-core email list history and this seems like the
> first patch from NI that has the tags included so I thought
> I'd reply and see if we can get the tags dropped.

IIRC, we pinged Phil a few months ago on this topic, and he thought
internal tags were OK. I think Wind River might have been cited as an
example.

I see that
http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
been updated with this requirement. Should it be? Will that require TSC
approval?

> -- 
> # Randy MacLeod. SMTS, Linux, Wind River
> Direct: 613.963.1350
> --
Richard Purdie - Aug. 23, 2014, 8:40 a.m.
On Fri, 2014-08-22 at 10:24 -0500, Richard Tollerton wrote:
> Randy MacLeod <randy.macleod@windriver.com> writes:
> 
> > Wind River patches used to include a "CQID" tag but we've changed
> > our process to avoid needing such internal tags. If National
> > Instruments can do so as well, that'd be best.
> >
> > I did check my oe-core email list history and this seems like the
> > first patch from NI that has the tags included so I thought
> > I'd reply and see if we can get the tags dropped.
> 
> IIRC, we pinged Phil a few months ago on this topic, and he thought
> internal tags were OK. I think Wind River might have been cited as an
> example.
> 
> I see that
> http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
> been updated with this requirement. Should it be? Will that require TSC
> approval?

I don't have a strong preference on this to be honest. I can imagine
cases where its useful to have some kind of tracking of issues into the
final commits. 

Obviously it would be better if everyone can understand what the numbers
mean, equally, it seems pointless to force people to strip them when
they might be useful to people contributing to the project.

If they are used as a substitute for a good commit message, that
wouldn't be acceptable. Also, if they were taking over the commit
messages, that would able be unacceptable. So if we can keep them to a
small part of the commit, I'm prepared to let them pass but it can't be
at the expense of good commit messages.

I don't believe we need a TSC decision, that would only be needed if
there were strong disagreements we were unable to resolve and I don't
think we're quite there yet :) I'll let others comment though and see
where we're at.

Cheers,

Richard
Paul Eggleton - Aug. 24, 2014, 3:33 p.m.
On Saturday 23 August 2014 09:40:44 Richard Purdie wrote:
> On Fri, 2014-08-22 at 10:24 -0500, Richard Tollerton wrote:
> > Randy MacLeod <randy.macleod@windriver.com> writes:
> > > Wind River patches used to include a "CQID" tag but we've changed
> > > our process to avoid needing such internal tags. If National
> > > Instruments can do so as well, that'd be best.
> > > 
> > > I did check my oe-core email list history and this seems like the
> > > first patch from NI that has the tags included so I thought
> > > I'd reply and see if we can get the tags dropped.
> > 
> > IIRC, we pinged Phil a few months ago on this topic, and he thought
> > internal tags were OK. I think Wind River might have been cited as an
> > example.
> > 
> > I see that
> > http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
> > been updated with this requirement. Should it be? Will that require TSC
> > approval?
> 
> I don't have a strong preference on this to be honest. I can imagine
> cases where its useful to have some kind of tracking of issues into the
> final commits.
> 
> Obviously it would be better if everyone can understand what the numbers
> mean, equally, it seems pointless to force people to strip them when
> they might be useful to people contributing to the project.
> 
> If they are used as a substitute for a good commit message, that
> wouldn't be acceptable. Also, if they were taking over the commit
> messages, that would able be unacceptable. So if we can keep them to a
> small part of the commit, I'm prepared to let them pass but it can't be
> at the expense of good commit messages.
> 
> I don't believe we need a TSC decision, that would only be needed if
> there were strong disagreements we were unable to resolve and I don't
> think we're quite there yet :) I'll let others comment though and see
> where we're at.

FWIW I agree with all of the above - if they make contributors' lives easier 
then I don't see a good reason to disallow their usage given we're only 
talking about one line at the end of a proper commit message.

Cheers,
Paul
Khem Raj - Aug. 25, 2014, 1:19 a.m.
On 14-08-24 16:33:39, Paul Eggleton wrote:
> On Saturday 23 August 2014 09:40:44 Richard Purdie wrote:
> > On Fri, 2014-08-22 at 10:24 -0500, Richard Tollerton wrote:
> > > Randy MacLeod <randy.macleod@windriver.com> writes:
> > > > Wind River patches used to include a "CQID" tag but we've changed
> > > > our process to avoid needing such internal tags. If National
> > > > Instruments can do so as well, that'd be best.
> > > > 
> > > > I did check my oe-core email list history and this seems like the
> > > > first patch from NI that has the tags included so I thought
> > > > I'd reply and see if we can get the tags dropped.
> > > 
> > > IIRC, we pinged Phil a few months ago on this topic, and he thought
> > > internal tags were OK. I think Wind River might have been cited as an
> > > example.
> > > 
> > > I see that
> > > http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
> > > been updated with this requirement. Should it be? Will that require TSC
> > > approval?
> > 
> > I don't have a strong preference on this to be honest. I can imagine
> > cases where its useful to have some kind of tracking of issues into the
> > final commits.
> > 
> > Obviously it would be better if everyone can understand what the numbers
> > mean, equally, it seems pointless to force people to strip them when
> > they might be useful to people contributing to the project.
> > 
> > If they are used as a substitute for a good commit message, that
> > wouldn't be acceptable. Also, if they were taking over the commit
> > messages, that would able be unacceptable. So if we can keep them to a
> > small part of the commit, I'm prepared to let them pass but it can't be
> > at the expense of good commit messages.
> > 
> > I don't believe we need a TSC decision, that would only be needed if
> > there were strong disagreements we were unable to resolve and I don't
> > think we're quite there yet :) I'll let others comment though and see
> > where we're at.
> 
> FWIW I agree with all of the above - if they make contributors' lives easier 
> then I don't see a good reason to disallow their usage given we're only 
> talking about one line at the end of a proper commit message.

I would rather think that if the information is not accessible to community
members, its not a useful to add these numbers. Secondly it could be fixing
someone else's bug in a different bug tracking system with different
tracking number and if he she does a backport and adds his tracking
number as well to commit. Now think of what will happen if people start
to do it for Linux kernel. So IMHO its best that companies track it in
their respective issue tracking systems by adding upstream commit urls or SHA ids
whatever is convinient to support any automation tools they might
have(if any) That way we solve the problem for everyone. On
these grounds I would not recommend adding it in commit messages. We let Yocto
bugzilla entries in commits because that system is accessible to everyone using OE.

Thanks

-Khem

Patch

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index f2c84d5..1e69861 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -69,7 +69,7 @@  case "$1" in
 		    readfiles /etc/udev/cache.data
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" ]; then
-                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
+                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
                             [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index db5a513..3769062 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -25,7 +25,7 @@  fi
 
 if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
 	echo "Populating dev cache"
-	(cd /; tar cf "$DEVCACHE" dev)
+	(cd /; tar czf "$DEVCACHE" dev)
 	mv /dev/shm/udev.cache /etc/udev/cache.data
 fi
 
diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 2093336..909ec87 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -1,5 +1,5 @@ 
 # Default for /etc/init.d/udev
 
 # Comment this out to disable device cache
-DEVCACHE="/etc/dev.tar"
+DEVCACHE="/etc/dev.tar.gz"
 PROBE_PLATFORM_BUS="yes"