| Message ID | 20230802214019.1068795-6-charlie.johnston@ni.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add new packagefeed recipe class. | expand |
Same comment as before: This does need to be tested, and it does need examples. Can you rework create_rpm_index() from meta/classes-recipe/testimage.bbclass to use this class instead of package-index? Alex On Wed, 2 Aug 2023 at 23:40, Charlie Johnston <charlie.johnston@ni.com> wrote: > > Add a new bbclass that allows building a feed using the > new oe.package_manager class. Additionally, there are > packagefeed_<type> bbclasses to define package type > specific configurations. > > The do_packagefeed task currently does no use SSTATE > data and is set to always run via [nostamp] = "1". > > The variable FEED_DEPENDS is used to specify feeds that > the packagefeed depends on and will be available > side-by-side. This prevents duplicate packages in the > two feeds. > > Signed-off-by: Charlie Johnston <charlie.johnston@ni.com> > --- > meta/classes-recipe/packagefeed.bbclass | 26 +++++++++++++++++++++ > meta/classes-recipe/packagefeed_deb.bbclass | 8 +++++++ > meta/classes-recipe/packagefeed_ipk.bbclass | 8 +++++++ > meta/classes-recipe/packagefeed_rpm.bbclass | 8 +++++++ > 4 files changed, 50 insertions(+) > create mode 100644 meta/classes-recipe/packagefeed.bbclass > create mode 100644 meta/classes-recipe/packagefeed_deb.bbclass > create mode 100644 meta/classes-recipe/packagefeed_ipk.bbclass > create mode 100644 meta/classes-recipe/packagefeed_rpm.bbclass > > diff --git a/meta/classes-recipe/packagefeed.bbclass b/meta/classes-recipe/packagefeed.bbclass > new file mode 100644 > index 0000000000..08df96f662 > --- /dev/null > +++ b/meta/classes-recipe/packagefeed.bbclass > @@ -0,0 +1,26 @@ > +# > +# Copyright OpenEmbedded Contributors > +# > +# SPDX-License-Identifier: MIT > +# > + > +PKGFEED_INHERITS = "${@' '.join(['packagefeed_' + x for x in d.getVar('PACKAGE_CLASSES').replace("package_", "").split()])}" > +inherit ${PKGFEED_INHERITS} nopackages > + > +LICENSE ?= "MIT" > + > +# Feeds listed in FEED_DEPENDS and their dependencies will be excluded from the feed. > +# This allows for side-by-side feeds without duplicate packages. > +FEED_DEPENDS ??= "" > + > +fakeroot python do_packagefeed() { > + from oe.package_manager import generate_feed_dirs, generate_index_files > + > + generate_feed_dirs(d, d.getVar("FEED_DEPENDS")) > + generate_index_files(d, isFeed=True) > +} > +addtask packagefeed before do_build > +do_packagefeed[recrdeptask] += "do_package_qa" > +do_packagefeed[nostamp] = "1" > +do_packagefeed[rdepends] += "${@' '.join([x + ':do_packagefeed' for x in d.getVar('FEED_DEPENDS').split()])}" > +do_packagefeed[cleandirs] += "${DEPLOY_DIR_FEED_DEB}/${PN} ${DEPLOY_DIR_FEED_IPK}/${PN} ${DEPLOY_DIR_FEED_RPM}/${PN}" > diff --git a/meta/classes-recipe/packagefeed_deb.bbclass b/meta/classes-recipe/packagefeed_deb.bbclass > new file mode 100644 > index 0000000000..2decc70a4f > --- /dev/null > +++ b/meta/classes-recipe/packagefeed_deb.bbclass > @@ -0,0 +1,8 @@ > +# > +# Copyright OpenEmbedded Contributors > +# > +# SPDX-License-Identifier: MIT > +# > + > +do_packagefeed[depends] += "apt-native:do_populate_sysroot" > +do_packagefeed[recrdeptask] += "do_package_write_deb" > diff --git a/meta/classes-recipe/packagefeed_ipk.bbclass b/meta/classes-recipe/packagefeed_ipk.bbclass > new file mode 100644 > index 0000000000..89d296200b > --- /dev/null > +++ b/meta/classes-recipe/packagefeed_ipk.bbclass > @@ -0,0 +1,8 @@ > +# > +# Copyright OpenEmbedded Contributors > +# > +# SPDX-License-Identifier: MIT > +# > + > +do_packagefeed[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot" > +do_packagefeed[recrdeptask] += "do_package_write_ipk" > diff --git a/meta/classes-recipe/packagefeed_rpm.bbclass b/meta/classes-recipe/packagefeed_rpm.bbclass > new file mode 100644 > index 0000000000..8ce37cc855 > --- /dev/null > +++ b/meta/classes-recipe/packagefeed_rpm.bbclass > @@ -0,0 +1,8 @@ > +# > +# Copyright OpenEmbedded Contributors > +# > +# SPDX-License-Identifier: MIT > +# > + > +do_packagefeed[depends] += "createrepo-c-native:do_populate_sysroot" > +do_packagefeed[recrdeptask] += "do_package_write_rpm" > -- > 2.41.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#185439): https://lists.openembedded.org/g/openembedded-core/message/185439 > Mute This Topic: https://lists.openembedded.org/mt/100515098/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 2 Aug 2023, at 22:35, Charlie Johnston via lists.openembedded.org <charlie.johnston=ni.com@lists.openembedded.org> wrote:
> +++ b/meta/classes-recipe/packagefeed.bbclass
So this is a recipe-scope class which adds a task to generate a package feed before do_build finishes. I imagine the usecase would be that you add the inherit to a packagegroup or image that defines the scope of the feed?
Personally I’ve often doing `bitbake package-index` to use the local feed on my target, but this class wouldn’t be useful because I’m not just building a single image and if I want to use the feed it’s often because I want to add a new package that wasn’t in the image before. All I want is a way to ensure that the deploy directory has an up to date feed in, am I right in thinking this class doesn’t solve my usecase?
Cheers,
Ross
On 8/3/23 13:32, Ross Burton wrote: > On 2 Aug 2023, at 22:35, Charlie Johnston via lists.openembedded.org <charlie.johnston=ni.com@lists.openembedded.org> wrote: >> +++ b/meta/classes-recipe/packagefeed.bbclass > > So this is a recipe-scope class which adds a task to generate a package feed before do_build finishes. I imagine the usecase would be that you add the inherit to a packagegroup or image that defines the scope of the feed? Currently the use case would be a new recipe defining the feed. That feed could simply contain a packagegroup, or multiple packages or packagegroups. I did consider leaving it up to the user to determine if the recipe were also a packagegroup or image or similar but I currently have packagefeed inherit nopackages. I could be convinced to go back on that. > Personally I’ve often doing `bitbake package-index` to use the local feed on my target, but this class wouldn’t be useful because I’m not just building a single image and if I want to use the feed it’s often because I want to add a new package that wasn’t in the image before. All I want is a way to ensure that the deploy directory has an up to date feed in, am I right in thinking this class doesn’t solve my usecase? > > Cheers, > Ross I don't think the changes address your use case as-is. If you used a packagefeed recipe to build all your packages and used that packagefeed's directory as the local feed, it might work but you would have to add the missing package to the packagefeed recipe (or a packagegroup in that recipe). I'm targeting the use cases where the feeds will be deployed publicly for use with images and (hopefully) for some future quality of life improvements to BUILD_IMAGES_FROM_FEEDS. I'm open to any ideas that might cover other uses. Thanks, Charlie Johnston
Got it. Is running runtime_test.TestImage.test_testimage_dnf sufficient for testing the change? Or are there specific images I should try with testimage? For the examples, how many sample recipes would be adequate? 1 or 2? Thanks, Charlie Johnston On 8/3/23 02:18, Alexander Kanavin wrote: > Same comment as before: > This does need to be tested, and it does need examples. Can you rework > create_rpm_index() from meta/classes-recipe/testimage.bbclass to use > this class instead of package-index? > > Alex > > On Wed, 2 Aug 2023 at 23:40, Charlie Johnston <charlie.johnston@ni.com> wrote: >> >> Add a new bbclass that allows building a feed using the >> new oe.package_manager class. Additionally, there are >> packagefeed_<type> bbclasses to define package type >> specific configurations. >> >> The do_packagefeed task currently does no use SSTATE >> data and is set to always run via [nostamp] = "1". >> >> The variable FEED_DEPENDS is used to specify feeds that >> the packagefeed depends on and will be available >> side-by-side. This prevents duplicate packages in the >> two feeds. >> >> Signed-off-by: Charlie Johnston <charlie.johnston@ni.com> >> --- >> meta/classes-recipe/packagefeed.bbclass | 26 +++++++++++++++++++++ >> meta/classes-recipe/packagefeed_deb.bbclass | 8 +++++++ >> meta/classes-recipe/packagefeed_ipk.bbclass | 8 +++++++ >> meta/classes-recipe/packagefeed_rpm.bbclass | 8 +++++++ >> 4 files changed, 50 insertions(+) >> create mode 100644 meta/classes-recipe/packagefeed.bbclass >> create mode 100644 meta/classes-recipe/packagefeed_deb.bbclass >> create mode 100644 meta/classes-recipe/packagefeed_ipk.bbclass >> create mode 100644 meta/classes-recipe/packagefeed_rpm.bbclass >> >> diff --git a/meta/classes-recipe/packagefeed.bbclass b/meta/classes-recipe/packagefeed.bbclass >> new file mode 100644 >> index 0000000000..08df96f662 >> --- /dev/null >> +++ b/meta/classes-recipe/packagefeed.bbclass >> @@ -0,0 +1,26 @@ >> +# >> +# Copyright OpenEmbedded Contributors >> +# >> +# SPDX-License-Identifier: MIT >> +# >> + >> +PKGFEED_INHERITS = "${@' '.join(['packagefeed_' + x for x in d.getVar('PACKAGE_CLASSES').replace("package_", "").split()])}" >> +inherit ${PKGFEED_INHERITS} nopackages >> + >> +LICENSE ?= "MIT" >> + >> +# Feeds listed in FEED_DEPENDS and their dependencies will be excluded from the feed. >> +# This allows for side-by-side feeds without duplicate packages. >> +FEED_DEPENDS ??= "" >> + >> +fakeroot python do_packagefeed() { >> + from oe.package_manager import generate_feed_dirs, generate_index_files >> + >> + generate_feed_dirs(d, d.getVar("FEED_DEPENDS")) >> + generate_index_files(d, isFeed=True) >> +} >> +addtask packagefeed before do_build >> +do_packagefeed[recrdeptask] += "do_package_qa" >> +do_packagefeed[nostamp] = "1" >> +do_packagefeed[rdepends] += "${@' '.join([x + ':do_packagefeed' for x in d.getVar('FEED_DEPENDS').split()])}" >> +do_packagefeed[cleandirs] += "${DEPLOY_DIR_FEED_DEB}/${PN} ${DEPLOY_DIR_FEED_IPK}/${PN} ${DEPLOY_DIR_FEED_RPM}/${PN}" >> diff --git a/meta/classes-recipe/packagefeed_deb.bbclass b/meta/classes-recipe/packagefeed_deb.bbclass >> new file mode 100644 >> index 0000000000..2decc70a4f >> --- /dev/null >> +++ b/meta/classes-recipe/packagefeed_deb.bbclass >> @@ -0,0 +1,8 @@ >> +# >> +# Copyright OpenEmbedded Contributors >> +# >> +# SPDX-License-Identifier: MIT >> +# >> + >> +do_packagefeed[depends] += "apt-native:do_populate_sysroot" >> +do_packagefeed[recrdeptask] += "do_package_write_deb" >> diff --git a/meta/classes-recipe/packagefeed_ipk.bbclass b/meta/classes-recipe/packagefeed_ipk.bbclass >> new file mode 100644 >> index 0000000000..89d296200b >> --- /dev/null >> +++ b/meta/classes-recipe/packagefeed_ipk.bbclass >> @@ -0,0 +1,8 @@ >> +# >> +# Copyright OpenEmbedded Contributors >> +# >> +# SPDX-License-Identifier: MIT >> +# >> + >> +do_packagefeed[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot" >> +do_packagefeed[recrdeptask] += "do_package_write_ipk" >> diff --git a/meta/classes-recipe/packagefeed_rpm.bbclass b/meta/classes-recipe/packagefeed_rpm.bbclass >> new file mode 100644 >> index 0000000000..8ce37cc855 >> --- /dev/null >> +++ b/meta/classes-recipe/packagefeed_rpm.bbclass >> @@ -0,0 +1,8 @@ >> +# >> +# Copyright OpenEmbedded Contributors >> +# >> +# SPDX-License-Identifier: MIT >> +# >> + >> +do_packagefeed[depends] += "createrepo-c-native:do_populate_sysroot" >> +do_packagefeed[recrdeptask] += "do_package_write_rpm" >> -- >> 2.41.0 >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#185439): https://lists.openembedded.org/g/openembedded-core/message/185439 >> Mute This Topic: https://lists.openembedded.org/mt/100515098/1686489 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >>
On Fri, 4 Aug 2023 at 01:53, Charlie Johnston <charlie.johnston@ni.com> wrote: > Got it. Is running runtime_test.TestImage.test_testimage_dnf sufficient for testing the change? Or are there specific images I should try with testimage? core-image-minimal, core-image-full-cmdline, core-image-sato, core-image-sato-sdk should do. > For the examples, how many sample recipes would be adequate? 1 or 2? I think just writing an example or two into the comment at the top of the class would be ok. Also, update the documentation, everywhere where package-index is mentioned: https://git.yoctoproject.org/yocto-docs/about/ Alex
diff --git a/meta/classes-recipe/packagefeed.bbclass b/meta/classes-recipe/packagefeed.bbclass new file mode 100644 index 0000000000..08df96f662 --- /dev/null +++ b/meta/classes-recipe/packagefeed.bbclass @@ -0,0 +1,26 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +PKGFEED_INHERITS = "${@' '.join(['packagefeed_' + x for x in d.getVar('PACKAGE_CLASSES').replace("package_", "").split()])}" +inherit ${PKGFEED_INHERITS} nopackages + +LICENSE ?= "MIT" + +# Feeds listed in FEED_DEPENDS and their dependencies will be excluded from the feed. +# This allows for side-by-side feeds without duplicate packages. +FEED_DEPENDS ??= "" + +fakeroot python do_packagefeed() { + from oe.package_manager import generate_feed_dirs, generate_index_files + + generate_feed_dirs(d, d.getVar("FEED_DEPENDS")) + generate_index_files(d, isFeed=True) +} +addtask packagefeed before do_build +do_packagefeed[recrdeptask] += "do_package_qa" +do_packagefeed[nostamp] = "1" +do_packagefeed[rdepends] += "${@' '.join([x + ':do_packagefeed' for x in d.getVar('FEED_DEPENDS').split()])}" +do_packagefeed[cleandirs] += "${DEPLOY_DIR_FEED_DEB}/${PN} ${DEPLOY_DIR_FEED_IPK}/${PN} ${DEPLOY_DIR_FEED_RPM}/${PN}" diff --git a/meta/classes-recipe/packagefeed_deb.bbclass b/meta/classes-recipe/packagefeed_deb.bbclass new file mode 100644 index 0000000000..2decc70a4f --- /dev/null +++ b/meta/classes-recipe/packagefeed_deb.bbclass @@ -0,0 +1,8 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +do_packagefeed[depends] += "apt-native:do_populate_sysroot" +do_packagefeed[recrdeptask] += "do_package_write_deb" diff --git a/meta/classes-recipe/packagefeed_ipk.bbclass b/meta/classes-recipe/packagefeed_ipk.bbclass new file mode 100644 index 0000000000..89d296200b --- /dev/null +++ b/meta/classes-recipe/packagefeed_ipk.bbclass @@ -0,0 +1,8 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +do_packagefeed[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot" +do_packagefeed[recrdeptask] += "do_package_write_ipk" diff --git a/meta/classes-recipe/packagefeed_rpm.bbclass b/meta/classes-recipe/packagefeed_rpm.bbclass new file mode 100644 index 0000000000..8ce37cc855 --- /dev/null +++ b/meta/classes-recipe/packagefeed_rpm.bbclass @@ -0,0 +1,8 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +do_packagefeed[depends] += "createrepo-c-native:do_populate_sysroot" +do_packagefeed[recrdeptask] += "do_package_write_rpm"
Add a new bbclass that allows building a feed using the new oe.package_manager class. Additionally, there are packagefeed_<type> bbclasses to define package type specific configurations. The do_packagefeed task currently does no use SSTATE data and is set to always run via [nostamp] = "1". The variable FEED_DEPENDS is used to specify feeds that the packagefeed depends on and will be available side-by-side. This prevents duplicate packages in the two feeds. Signed-off-by: Charlie Johnston <charlie.johnston@ni.com> --- meta/classes-recipe/packagefeed.bbclass | 26 +++++++++++++++++++++ meta/classes-recipe/packagefeed_deb.bbclass | 8 +++++++ meta/classes-recipe/packagefeed_ipk.bbclass | 8 +++++++ meta/classes-recipe/packagefeed_rpm.bbclass | 8 +++++++ 4 files changed, 50 insertions(+) create mode 100644 meta/classes-recipe/packagefeed.bbclass create mode 100644 meta/classes-recipe/packagefeed_deb.bbclass create mode 100644 meta/classes-recipe/packagefeed_ipk.bbclass create mode 100644 meta/classes-recipe/packagefeed_rpm.bbclass