Patchwork [meta-oe,6/7] systemd.bbclass: automatically extend FILES_* for systemd packages

login
register
mail settings
Submitter Andreas Müller
Date Feb. 11, 2012, 2 a.m.
Message ID <1328925603-2967-6-git-send-email-schnitzeltony@googlemail.com>
Download mbox | patch
Permalink /patch/21165/
State Superseded
Headers show

Comments

Andreas Müller - Feb. 11, 2012, 2 a.m.
In case SYSTEMD_SERVICE contains only one *.service file, the whole folder
${base_libdir}/systemd/system/ is added to FILES_* otherwise only the *.service
files found in SYSTEMD_SERVICE.

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
---
 meta-oe/classes/systemd.bbclass |   16 ++++++++++++++++
 1 files changed, 16 insertions(+), 0 deletions(-)
Otavio Salvador - Feb. 11, 2012, 1:12 p.m.
On Sat, Feb 11, 2012 at 00:00, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> In case SYSTEMD_SERVICE contains only one *.service file, the whole folder
> ${base_libdir}/systemd/system/ is added to FILES_* otherwise only the
> *.service
> files found in SYSTEMD_SERVICE.
>

It seems you'll need to parse the service file and check if it installs
others. For example .socket files usually do that. And then you'll never
add the whole dir automatically but dependently of the services it needs to
install.
Andreas Müller - Feb. 12, 2012, 10:15 p.m.
On Sat, Feb 11, 2012 at 2:12 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Sat, Feb 11, 2012 at 00:00, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> In case SYSTEMD_SERVICE contains only one *.service file, the whole folder
>> ${base_libdir}/systemd/system/ is added to FILES_* otherwise only the
>> *.service
>> files found in SYSTEMD_SERVICE.
>>
Off topic: My intention was to install the whole directory for recipes
with one service per package AND one *-systemd package (this case is
not catched by my code - so there will be a V2)
>
> It seems you'll need to parse the service file and check if it installs
> others. For example .socket files usually do that. And then you'll never
> add the whole dir automatically but dependently of the services it needs to
> install.
>
I am afraid this goes a bit to far and causes confusion what is
packed. I would prefer the simple rule:

If there is one *-systemd package with one entry in SYSTEMD_SERVICE,
the whole directory is installed - otherwise only the files found in
SYSTEMD_SERVICE. If a recipe installs too much or something is
missing, it can be corrected in recipe.

What is common opinion here?

Andreas
Otavio Salvador - Feb. 13, 2012, 10:42 a.m.
On Sun, Feb 12, 2012 at 20:15, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> If there is one *-systemd package with one entry in SYSTEMD_SERVICE,
> the whole directory is installed - otherwise only the files found in
> SYSTEMD_SERVICE. If a recipe installs too much or something is
> missing, it can be corrected in recipe.
>

If for example I put two .socket files, your algorithm will break. That's
why I said you need to parse it. It is not that difficult as the logic is
the same as the wrapper used in rootfs generation.
Andreas Müller - Feb. 13, 2012, 12:05 p.m.
On Mon, Feb 13, 2012 at 11:42 AM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Sun, Feb 12, 2012 at 20:15, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> If there is one *-systemd package with one entry in SYSTEMD_SERVICE,
>> the whole directory is installed - otherwise only the files found in
>> SYSTEMD_SERVICE. If a recipe installs too much or something is
>> missing, it can be corrected in recipe.
>>
>
> If for example I put two .socket files, your algorithm will break. That's
> why I said you need to parse it. It is not that difficult as the logic is
> the same as the wrapper used in rootfs generation.
Sorry - but which which wrapper do you mean?

Andreas
Otavio Salvador - Feb. 13, 2012, 12:14 p.m.
On Mon, Feb 13, 2012 at 10:05, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> > If for example I put two .socket files, your algorithm will break. That's
> > why I said you need to parse it. It is not that difficult as the logic is
> > the same as the wrapper used in rootfs generation.
> Sorry - but which which wrapper do you mean?


systemd-systemctl-native
Andreas Müller - Feb. 13, 2012, 4:38 p.m.
On Mon, Feb 13, 2012 at 1:14 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Mon, Feb 13, 2012 at 10:05, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> > If for example I put two .socket files, your algorithm will break. That's
>> > why I said you need to parse it. It is not that difficult as the logic is
>> > the same as the wrapper used in rootfs generation.
>> Sorry - but which which wrapper do you mean?
>
>
> systemd-systemctl-native
Off list @Otavio: not boring but challenging...

To reuse as much as possible: Is there a piece of code showing me how
to mix up shell/python code?

Andreas
Otavio Salvador - Feb. 13, 2012, 5:09 p.m.
On Mon, Feb 13, 2012 at 14:38, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> > systemd-systemctl-native
> Off list @Otavio: not boring but challenging...
>
> To reuse as much as possible: Is there a piece of code showing me how
> to mix up shell/python code?


Not that I am aware of but you can look at combo-layer script how to read
and parse ini files; this ought to be very simple to do.
Andreas Müller - Feb. 14, 2012, 12:42 p.m.
On Mon, Feb 13, 2012 at 6:09 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Mon, Feb 13, 2012 at 14:38, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> > systemd-systemctl-native
>> Off list @Otavio: not boring but challenging...
>>
>> To reuse as much as possible: Is there a piece of code showing me how
>> to mix up shell/python code?
>
>
> Not that I am aware of but you can look at combo-layer script how to read
> and parse ini files; this ought to be very simple to do.
>
Currently I am testing the automatic parsing for different recipes.
Following the 'After=' and 'WantedBy=' works fine but before sending
out I would like to get some comments on the following
assumptions/questions:

* There is no need for recipes to install links for 'WantedBy' entries
since systemd-systemctl-native takes care. E.G the code in
lightttpd_1.4.30.bbappend for this can be removed (it works on the
wrong paths anyway) -> there is no need for systemd.bbclass to follow
'WantedBy'. Correct?

* For dropbear_2011.54.bbappend, the automatic parsing leaves
dropbear@.service and dropbear.service ( link to /dev/null ) unpacked.
In dropbear.socket there is 'Conflicts=dropbear.service'. Shall the
algorithm follow also services linked by Conflicts= and pack those
service files?

* dropbear_2011.54.bbappend: Is dropbear@.service a candidate for a
second *-systemd packet?

* are there other keys which might need follow-up too?

Thanks in advance

Andreas
Otavio Salvador - Feb. 14, 2012, 12:51 p.m.
On Tue, Feb 14, 2012 at 10:42, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> Currently I am testing the automatic parsing for different recipes.
> Following the 'After=' and 'WantedBy=' works fine but before sending
> out I would like to get some comments on the following
> assumptions/questions:
>
> * There is no need for recipes to install links for 'WantedBy' entries
> since systemd-systemctl-native takes care. E.G the code in
> lightttpd_1.4.30.bbappend for this can be removed (it works on the
> wrong paths anyway) -> there is no need for systemd.bbclass to follow
> 'WantedBy'. Correct?
>

Links ought to be kept to be handled by systemd-systemctl-native.


> * For dropbear_2011.54.bbappend, the automatic parsing leaves
> dropbear@.service and dropbear.service ( link to /dev/null ) unpacked.
> In dropbear.socket there is 'Conflicts=dropbear.service'. Shall the
> algorithm follow also services linked by Conflicts= and pack those
> service files?
>

The /dev/null link is to force disable the init script. This is difficult
to do automatically so for now I'd say to do it byhand.


> * dropbear_2011.54.bbappend: Is dropbear@.service a candidate for a
> second *-systemd packet?
>

No; The @ is to allow multiple instance. Look at
http://www.freedesktop.org/software/systemd/man/systemd.unit.html


> * are there other keys which might need follow-up too?
>

Not that I am aware of at this moment.
Andreas Müller - Feb. 14, 2012, 4:09 p.m.
On Tue, Feb 14, 2012 at 1:51 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> No; The @ is to allow multiple instance. Look at
> http://www.freedesktop.org/software/systemd/man/systemd.unit.html
How about the old strategy for *@.service:
If there is only one *-systemd package with one entry in
SYSTEMD_SERVICE it will get *@.service?

Andreas
Otavio Salvador - Feb. 14, 2012, 4:22 p.m.
On Tue, Feb 14, 2012 at 14:09, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> On Tue, Feb 14, 2012 at 1:51 PM, Otavio Salvador
> <otavio@ossystems.com.br> wrote:
> > No; The @ is to allow multiple instance. Look at
> > http://www.freedesktop.org/software/systemd/man/systemd.unit.html
> How about the old strategy for *@.service:
> If there is only one *-systemd package with one entry in
> SYSTEMD_SERVICE it will get *@.service?


I think so.

It is not easy to predict all possible usages but we ought to take this
route and adjust it one the way. Once you do it and we start converting all
recipes that use systemd class in meta-oe we'll most probably find many
possible corner-cases to deal with.
Andreas Müller - Feb. 15, 2012, 12:31 p.m.
On Tue, Feb 14, 2012 at 5:22 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Tue, Feb 14, 2012 at 14:09, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> On Tue, Feb 14, 2012 at 1:51 PM, Otavio Salvador
>> <otavio@ossystems.com.br> wrote:
>> > No; The @ is to allow multiple instance. Look at
>> > http://www.freedesktop.org/software/systemd/man/systemd.unit.html
>> How about the old strategy for *@.service:
>> If there is only one *-systemd package with one entry in
>> SYSTEMD_SERVICE it will get *@.service?
>
>
> I think so.
>
> It is not easy to predict all possible usages but we ought to take this
> route and adjust it one the way. Once you do it and we start converting all
> recipes that use systemd class in meta-oe we'll most probably find many
> possible corner-cases to deal with.
>
Another corner case I would like to understand: What is the role of in
busybox syslog.service linking to dev/null. Within busybox itself it
seems not referenced.

Thanks in advance

Andreas
Otavio Salvador - Feb. 15, 2012, 12:36 p.m.
On Wed, Feb 15, 2012 at 10:31, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> Another corner case I would like to understand: What is the role of in
> busybox syslog.service linking to dev/null. Within busybox itself it
> seems not referenced.
>

To make it to use the .service file and not the init scripts compatibility.
Andreas Müller - Feb. 15, 2012, 7:53 p.m.
On Wed, Feb 15, 2012 at 1:36 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Wed, Feb 15, 2012 at 10:31, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> Another corner case I would like to understand: What is the role of in
>> busybox syslog.service linking to dev/null. Within busybox itself it
>> seems not referenced.
>>
>
> To make it to use the .service file and not the init scripts compatibility.
>
For test I did with a bunch of recipes

1. build from scratch with unmodified sources
2. Applied sytemd-patches and the 'cleanup'-patches (removing obsolete
systemd code in recipes)
3. build recipes again
4. compared the results with buildhistory

Results are as expected except one issue I don't understand:

The sizes of the *-dbg packages are reduced. Some reduce to less than
half. Has anybody a good explanation for this?

Andreas
Otavio Salvador - Feb. 15, 2012, 7:55 p.m.
On Wed, Feb 15, 2012 at 17:53, Andreas Müller
<schnitzeltony@googlemail.com>wrote:

> For test I did with a bunch of recipes
>
> 1. build from scratch with unmodified sources
> 2. Applied sytemd-patches and the 'cleanup'-patches (removing obsolete
> systemd code in recipes)
> 3. build recipes again
> 4. compared the results with buildhistory
>
> Results are as expected except one issue I don't understand:
>
> The sizes of the *-dbg packages are reduced. Some reduce to less than
> half. Has anybody a good explanation for this?


No good guess about possible reasoning.
Martin Jansa - Feb. 15, 2012, 8:05 p.m.
On Wed, Feb 15, 2012 at 05:55:56PM -0200, Otavio Salvador wrote:
> On Wed, Feb 15, 2012 at 17:53, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
> 
> > For test I did with a bunch of recipes
> >
> > 1. build from scratch with unmodified sources
> > 2. Applied sytemd-patches and the 'cleanup'-patches (removing obsolete
> > systemd code in recipes)
> > 3. build recipes again
> > 4. compared the results with buildhistory
> >
> > Results are as expected except one issue I don't understand:
> >
> > The sizes of the *-dbg packages are reduced. Some reduce to less than
> > half. Has anybody a good explanation for this?
> 
> 
> No good guess about possible reasoning.

Just guess:

-dbg packages contain also sources packaged for /usr/src/debug/, maybe
your changes caused *.service files to be ommited from it?

Cheers,
Andreas Müller - Feb. 15, 2012, 8:07 p.m.
On Wed, Feb 15, 2012 at 8:55 PM, Otavio Salvador
<otavio@ossystems.com.br> wrote:
> On Wed, Feb 15, 2012 at 17:53, Andreas Müller
> <schnitzeltony@googlemail.com>wrote:
>
>> For test I did with a bunch of recipes
>>
>> 1. build from scratch with unmodified sources
>> 2. Applied sytemd-patches and the 'cleanup'-patches (removing obsolete
>> systemd code in recipes)
>> 3. build recipes again
>> 4. compared the results with buildhistory
>>
>> Results are as expected except one issue I don't understand:
>>
>> The sizes of the *-dbg packages are reduced. Some reduce to less than
>> half. Has anybody a good explanation for this?
>
>
> No good guess about possible reasoning.
>
I did further checks: seems many many source code files are missing
(example - see attachment). Strange is that the value for FILES has
not changed!  What have done to cause that - or is this unrelated to
my modifications?

Andreas
Andreas Müller - Feb. 15, 2012, 8:18 p.m.
On Wed, Feb 15, 2012 at 9:07 PM, Andreas Müller
<schnitzeltony@googlemail.com> wrote:
> On Wed, Feb 15, 2012 at 8:55 PM, Otavio Salvador
> <otavio@ossystems.com.br> wrote:
>> On Wed, Feb 15, 2012 at 17:53, Andreas Müller
>> <schnitzeltony@googlemail.com>wrote:
>>
>>> For test I did with a bunch of recipes
>>>
>>> 1. build from scratch with unmodified sources
>>> 2. Applied sytemd-patches and the 'cleanup'-patches (removing obsolete
>>> systemd code in recipes)
>>> 3. build recipes again
>>> 4. compared the results with buildhistory
>>>
>>> Results are as expected except one issue I don't understand:
>>>
>>> The sizes of the *-dbg packages are reduced. Some reduce to less than
>>> half. Has anybody a good explanation for this?
>>
>>
>> No good guess about possible reasoning.
>>
> I did further checks: seems many many source code files are missing
> (example - see attachment). Strange is that the value for FILES has
> not changed!  What have done to cause that - or is this unrelated to
> my modifications?
>
*All* source files are missing. Which part of oe is responsible, packing these?

Andreas
Andreas Müller - Feb. 15, 2012, 8:39 p.m.
On Wed, Feb 15, 2012 at 9:18 PM, Andreas Müller
<schnitzeltony@googlemail.com> wrote:
> On Wed, Feb 15, 2012 at 9:07 PM, Andreas Müller
> <schnitzeltony@googlemail.com> wrote:
>> On Wed, Feb 15, 2012 at 8:55 PM, Otavio Salvador
>> <otavio@ossystems.com.br> wrote:
>>> On Wed, Feb 15, 2012 at 17:53, Andreas Müller
>>> <schnitzeltony@googlemail.com>wrote:
>>>
>>>> For test I did with a bunch of recipes
>>>>
>>>> 1. build from scratch with unmodified sources
>>>> 2. Applied sytemd-patches and the 'cleanup'-patches (removing obsolete
>>>> systemd code in recipes)
>>>> 3. build recipes again
>>>> 4. compared the results with buildhistory
>>>>
>>>> Results are as expected except one issue I don't understand:
>>>>
>>>> The sizes of the *-dbg packages are reduced. Some reduce to less than
>>>> half. Has anybody a good explanation for this?
>>>
>>>
>>> No good guess about possible reasoning.
>>>
>> I did further checks: seems many many source code files are missing
>> (example - see attachment). Strange is that the value for FILES has
>> not changed!  What have done to cause that - or is this unrelated to
>> my modifications?
>>
> *All* source files are missing. Which part of oe is responsible, packing these?
>
OK I think this one unrelated to my changes. I did

* undo my patches
* bumped PR for poppler ( this one has no systemd 'contact' )
* build poppler

and for the bumped package

<..>/package/src/debug/

is empty!

Could somebody of you please do same and confirm?

Andreas
Andreas Müller - Feb. 15, 2012, 9 p.m.
On Wed, Feb 15, 2012 at 9:39 PM, Andreas Müller
<schnitzeltony@googlemail.com> wrote:
> <..>/package/src/debug/

of course this must be

<..>/package/usr/src/debug/

started a build from scratch with systemd patches...

Andreas

Patch

diff --git a/meta-oe/classes/systemd.bbclass b/meta-oe/classes/systemd.bbclass
index 3c1ed14..8927980 100644
--- a/meta-oe/classes/systemd.bbclass
+++ b/meta-oe/classes/systemd.bbclass
@@ -120,7 +120,23 @@  python populate_packages_prepend () {
 		rdepends.append("systemd")
 		bb.data.setVar('RDEPENDS_' + pkg, " " + " ".join(rdepends), d)
 
+	# add systemd files to FILES_*-systemd
+	def systemd_add_files(pkg_systemd):
+		systemd_services = d.getVar('SYSTEMD_SERVICE' + "_" + pkg_systemd, 1) or d.getVar('SYSTEMD_SERVICE', 1)
+		files_append = ""
+		if len(systemd_services.split()) == 1:
+			# distribute complete ${base_libdir}/systemd/system/ to ${SYSTEMD_PACKAGES}
+			files_append = '${base_libdir}/systemd/system/'
+		else:
+			# distribute files set in SYSTEMD_SERVICE to ${SYSTEMD_PACKAGES}
+			for service in systemd_services.split():
+				files_append += ' ${base_libdir}/systemd/system/' + service
+
+		var_name = "FILES_" + pkg_systemd
+		files = d.getVar(var_name, 0) or ""
+		d.setVar(var_name, "%s %s" % (files, files_append))
 
 	for pkg_systemd in d.getVar('SYSTEMD_PACKAGES', 1).split():
 		systemd_prepost_instrm(pkg_systemd)
+		systemd_add_files(pkg_systemd)
 }