Patchwork bitbake.conf: Add git-native to ASSUME_PROVIDED

login
register
mail settings
Submitter Richard Purdie
Date July 10, 2012, 1:32 p.m.
Message ID <1341927124.24837.8.camel@ted>
Download mbox | patch
Permalink /patch/31631/
State Accepted
Commit 79e24186481770181565a18d177584d0d72399fe
Headers show

Comments

Richard Purdie - July 10, 2012, 1:32 p.m.
Originally, git was something new, not installed everywhere and had commandline
stability problems. This has changed and git it no longer makes sense to
continually build this when the system installed version is likely sufficient.

This speeds up build since recipes no longer have to wait for git-native to build
if they're fetched from a git:// SRC_URI.

Also add git to the sanity checks and drop the no unneeded svn reference.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
Robert P. J. Day - July 10, 2012, 2:02 p.m.
On Tue, 10 Jul 2012, Richard Purdie wrote:

> Originally, git was something new, not installed everywhere and had commandline
> stability problems. This has changed and git it no longer makes sense to
> continually build this when the system installed version is likely sufficient.
>
> This speeds up build since recipes no longer have to wait for git-native to build
> if they're fetched from a git:// SRC_URI.
>
> Also add git to the sanity checks and drop the no unneeded svn reference.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 6ed1e6f..ffd59e9 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
> @@ -2,7 +2,7 @@
>  # Sanity check the users setup for common misconfigurations
>  #
>
> -SANITY_REQUIRED_UTILITIES ?= "patch diffstat texi2html makeinfo svn bzip2 tar gzip gawk chrpath wget cpio"
> +SANITY_REQUIRED_UTILITIES ?= "patch diffstat texi2html makeinfo git bzip2 tar gzip gawk chrpath wget cpio"
>
>  def raise_sanity_error(msg, d):
>      if d.getVar("SANITY_USE_EVENTS", True) == "1":
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 76ee65f..c94012e 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -152,6 +152,7 @@ DATETIME = "${DATE}${TIME}"
>  # its own in staging
>  ASSUME_PROVIDED = "\
>      bzip2-native \
> +    git-native \
>      grep-native \
>      diffstat-native \
>      patch-native \

  possibly a dumb question, but is this the sort of thing one could
extend in one's local.conf file if they were fairly sure their
installed versions were compatible?

rday
Richard Purdie - July 10, 2012, 2:09 p.m.
On Tue, 2012-07-10 at 10:02 -0400, Robert P. J. Day wrote:
> On Tue, 10 Jul 2012, Richard Purdie wrote:
> 
> > Originally, git was something new, not installed everywhere and had commandline
> > stability problems. This has changed and git it no longer makes sense to
> > continually build this when the system installed version is likely sufficient.
> >
> > This speeds up build since recipes no longer have to wait for git-native to build
> > if they're fetched from a git:// SRC_URI.
> >
> > Also add git to the sanity checks and drop the no unneeded svn reference.
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> > index 6ed1e6f..ffd59e9 100644
> > --- a/meta/classes/sanity.bbclass
> > +++ b/meta/classes/sanity.bbclass
> > @@ -2,7 +2,7 @@
> >  # Sanity check the users setup for common misconfigurations
> >  #
> >
> > -SANITY_REQUIRED_UTILITIES ?= "patch diffstat texi2html makeinfo svn bzip2 tar gzip gawk chrpath wget cpio"
> > +SANITY_REQUIRED_UTILITIES ?= "patch diffstat texi2html makeinfo git bzip2 tar gzip gawk chrpath wget cpio"
> >
> >  def raise_sanity_error(msg, d):
> >      if d.getVar("SANITY_USE_EVENTS", True) == "1":
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 76ee65f..c94012e 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -152,6 +152,7 @@ DATETIME = "${DATE}${TIME}"
> >  # its own in staging
> >  ASSUME_PROVIDED = "\
> >      bzip2-native \
> > +    git-native \
> >      grep-native \
> >      diffstat-native \
> >      patch-native \
> 
>   possibly a dumb question, but is this the sort of thing one could
> extend in one's local.conf file if they were fairly sure their
> installed versions were compatible?

For some things like git-native, it would be safe. For others like
perl-native or python-native, you could seriously shoot yourself in the
foot. subversion-native needs to be 1.7+ as an example.

The general idea is correct though, you can extend this from local.conf,
yes.

Cheers,

Richard
Robert P. J. Day - July 10, 2012, 2:14 p.m.
On Tue, 10 Jul 2012, Richard Purdie wrote:

> On Tue, 2012-07-10 at 10:02 -0400, Robert P. J. Day wrote:
> > On Tue, 10 Jul 2012, Richard Purdie wrote:

> > >  # its own in staging
> > >  ASSUME_PROVIDED = "\
> > >      bzip2-native \
> > > +    git-native \
> > >      grep-native \
> > >      diffstat-native \
> > >      patch-native \
> >
> >   possibly a dumb question, but is this the sort of thing one could
> > extend in one's local.conf file if they were fairly sure their
> > installed versions were compatible?
>
> For some things like git-native, it would be safe. For others like
> perl-native or python-native, you could seriously shoot yourself in
> the foot. subversion-native needs to be 1.7+ as an example.
>
> The general idea is correct though, you can extend this from
> local.conf, yes.

  sure, i appreciate that there's always danger, but with a current
distro, one would think that most standard utilities would be new
enough to work just fine.  thanks.

rday
McClintock Matthew-B29882 - July 10, 2012, 3 p.m.
On Tue, Jul 10, 2012 at 8:32 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Originally, git was something new, not installed everywhere and had commandline
> stability problems. This has changed and git it no longer makes sense to
> continually build this when the system installed version is likely sufficient.
>
> This speeds up build since recipes no longer have to wait for git-native to build
> if they're fetched from a git:// SRC_URI.
>
> Also add git to the sanity checks and drop the no unneeded svn reference.

What about testing the host system for presence of certain things?
Then one could

1) yum/apt/etc install the needed items

or

2) Build it if it's not present

-M
Richard Purdie - July 10, 2012, 3:57 p.m.
On Tue, 2012-07-10 at 15:00 +0000, McClintock Matthew-B29882 wrote:
> On Tue, Jul 10, 2012 at 8:32 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Originally, git was something new, not installed everywhere and had commandline
> > stability problems. This has changed and git it no longer makes sense to
> > continually build this when the system installed version is likely sufficient.
> >
> > This speeds up build since recipes no longer have to wait for git-native to build
> > if they're fetched from a git:// SRC_URI.
> >
> > Also add git to the sanity checks and drop the no unneeded svn reference.
> 
> What about testing the host system for presence of certain things?
> Then one could
> 
> 1) yum/apt/etc install the needed items
> 
> or
> 
> 2) Build it if it's not present

Every time we put an if statement in like this, it basically means we
should double the amount of QA, one for one path and one for the other.

So whilst I've wondered about it, do we really need to do something like
this?

Cheers,

Richard
Ross Burton - July 10, 2012, 4:06 p.m.
On 10 July 2012 16:57, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>> What about testing the host system for presence of certain things?
>> Then one could
>>
>> 1) yum/apt/etc install the needed items
>>
>> or
>>
>> 2) Build it if it's not present
>
> Every time we put an if statement in like this, it basically means we
> should double the amount of QA, one for one path and one for the other.
>
> So whilst I've wondered about it, do we really need to do something like
> this?

Back in the day I used to massively set ASSUME_PROVIDED in local.conf
to reduce the amount of -native packages built.  When it works it's
wonderful, but then you start getting mysterious failures because it
turns out that your system's tool has different build options, or is
out of date as Poky updated and you didn't notice.

It turned out to be a false economy: by the time I'd installed and
verified the packages from Debian I could have had Poky build them for
me. :)

Ross
McClintock Matthew-B29882 - July 10, 2012, 4:11 p.m.
On Tue, Jul 10, 2012 at 10:57 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Every time we put an if statement in like this, it basically means we
> should double the amount of QA, one for one path and one for the other.
>
> So whilst I've wondered about it, do we really need to do something like
> this?

I don't think we need it, it just has the possibility of improving the
initial user experience. What's the QA difference in testing all the
various differences in git (or whatever) on the distros we intend to
support? In a way ASSUME_PROVIDED += "git" more than doubled the QA ;)

-M
Robert P. J. Day - July 10, 2012, 6:35 p.m.
On Tue, 10 Jul 2012, Richard Purdie wrote:

> On Tue, 2012-07-10 at 15:00 +0000, McClintock Matthew-B29882 wrote:
> > On Tue, Jul 10, 2012 at 8:32 AM, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > Originally, git was something new, not installed everywhere and had commandline
> > > stability problems. This has changed and git it no longer makes sense to
> > > continually build this when the system installed version is likely sufficient.
> > >
> > > This speeds up build since recipes no longer have to wait for git-native to build
> > > if they're fetched from a git:// SRC_URI.
> > >
> > > Also add git to the sanity checks and drop the no unneeded svn reference.
> >
> > What about testing the host system for presence of certain things?
> > Then one could
> >
> > 1) yum/apt/etc install the needed items
> >
> > or
> >
> > 2) Build it if it's not present
>
> Every time we put an if statement in like this, it basically means we
> should double the amount of QA, one for one path and one for the other.
>
> So whilst I've wondered about it, do we really need to do something like
> this?

  my opinion, which is mine, is to add a script someone can run that
tells them what appears to be safe to override from their current
distro.  that way, it's entirely optional and the developer deals with
the output at their own risk.

rday
Khem Raj - July 10, 2012, 6:52 p.m.
On Tue, Jul 10, 2012 at 11:35 AM, Robert P. J. Day
<rpjday@crashcourse.ca> wrote:
>
>   my opinion, which is mine, is to add a script someone can run that
> tells them what appears to be safe to override from their current
> distro.  that way, it's entirely optional and the developer deals with
> the output at their own risk.

who determines the safeness ? thats what double QA RP mentioned is
Richard Purdie - July 10, 2012, 7:23 p.m.
On Tue, 2012-07-10 at 11:52 -0700, Khem Raj wrote:
> On Tue, Jul 10, 2012 at 11:35 AM, Robert P. J. Day
> <rpjday@crashcourse.ca> wrote:
> >
> >   my opinion, which is mine, is to add a script someone can run that
> > tells them what appears to be safe to override from their current
> > distro.  that way, it's entirely optional and the developer deals with
> > the output at their own risk.
> 
> who determines the safeness ? thats what double QA RP mentioned is

If a script says its safe, the bug reports will come in about the script
being broken.

Most -native dependencies are either essential (perl, python, dbus,
autoconf, automake, libtool, gmp, mpc, mfpr) or trivial. The ones that I
really dislike are the ones that hold up the critical path of the build
(i.e. libc/toolchain).

git-native is now falling into this category. subversion-native used to
be an issue but is rapidly being consigned to the history books.

So bang for buck, git-native makes sense, most other -native
dependencies are required at this point. The only two I'd "safely" put
in an ASSUME_PROVIDED locally and off the top of my head are flex/bison
at this point and those are questionable with some known API gotchas.
We've even had problems with things like tar recently. Hard to believe
but true and we have the bug reports to prove it.

Cheers,

Richard
Robert P. J. Day - July 10, 2012, 7:58 p.m.
On Tue, 10 Jul 2012, Khem Raj wrote:

> On Tue, Jul 10, 2012 at 11:35 AM, Robert P. J. Day
> <rpjday@crashcourse.ca> wrote:
> >
> >   my opinion, which is mine, is to add a script someone can run that
> > tells them what appears to be safe to override from their current
> > distro.  that way, it's entirely optional and the developer deals with
> > the output at their own risk.
>
> who determines the safeness ? thats what double QA RP mentioned is

  there's no *guarantee* of safeness, just an advisory based on
nothing more than the version number.  user makes their own decision
and, if it breaks, they get to keep all the pieces.

rday
Enrico Scholz - July 11, 2012, 12:23 p.m.
Richard Purdie
<richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
writes:

> Originally, git was something new, not installed everywhere and had
> commandline stability problems. This has changed and git it no longer
> makes sense to continually build this when the system installed version
> is likely sufficient.

Can this really be assumed? Recent bitbake use something like 

 | git remote add --mirror=fetch

The git version in RHEL 6 or Ubuntu 10 are too old for this and support
'--mirror' without a value only.


Enrico
Richard Purdie - July 11, 2012, 4:19 p.m.
On Wed, 2012-07-11 at 14:23 +0200, Enrico Scholz wrote:
> Richard Purdie
> <richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
> writes:
> 
> > Originally, git was something new, not installed everywhere and had
> > commandline stability problems. This has changed and git it no longer
> > makes sense to continually build this when the system installed version
> > is likely sufficient.
> 
> Can this really be assumed? Recent bitbake use something like 
> 
>  | git remote add --mirror=fetch
> 
> The git version in RHEL 6 or Ubuntu 10 are too old for this and support
> '--mirror' without a value only.

You're right unfortunately, we're going to need to rethink this. I
didn't think the git fetcher used any advanced syntax now but it clearly
does :(

Does anyone know what the actual version dependency is? I know its after
1.7.3.4 and before 1.7.5.4.

Cheers,

Richard
Colin Walters - July 16, 2012, 10:06 p.m.
On Tue, 2012-07-10 at 15:00 +0000, McClintock Matthew-B29882 wrote:
> On Tue, Jul 10, 2012 at 8:32 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Originally, git was something new, not installed everywhere and had commandline
> > stability problems. This has changed and git it no longer makes sense to
> > continually build this when the system installed version is likely sufficient.
> >
> > This speeds up build since recipes no longer have to wait for git-native to build
> > if they're fetched from a git:// SRC_URI.
> >
> > Also add git to the sanity checks and drop the no unneeded svn reference.
> 
> What about testing the host system for presence of certain things?
> Then one could
> 
> 1) yum/apt/etc install the needed items

We do something like this in GNOME/jhbuild, but there's no version
comparison:

https://bugzilla.gnome.org/show_bug.cgi?id=564373

This is mainly targeted for system libraries which have pkg-config
files, but we're discussing extending it to executables and
non-pkg-config headers:

https://bugzilla.gnome.org/show_bug.cgi?id=671042

Patch

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index 6ed1e6f..ffd59e9 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -2,7 +2,7 @@ 
 # Sanity check the users setup for common misconfigurations
 #
 
-SANITY_REQUIRED_UTILITIES ?= "patch diffstat texi2html makeinfo svn bzip2 tar gzip gawk chrpath wget cpio"
+SANITY_REQUIRED_UTILITIES ?= "patch diffstat texi2html makeinfo git bzip2 tar gzip gawk chrpath wget cpio"
 
 def raise_sanity_error(msg, d):
     if d.getVar("SANITY_USE_EVENTS", True) == "1":
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 76ee65f..c94012e 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -152,6 +152,7 @@  DATETIME = "${DATE}${TIME}"
 # its own in staging
 ASSUME_PROVIDED = "\
     bzip2-native \
+    git-native \
     grep-native \
     diffstat-native \
     patch-native \