| Message ID | 20230719200252.1203047-1-charlie.johnston@ni.com |
|---|---|
| Headers | show |
| Series | Add packagefeed recipe class | expand |
You need to start by describing the use case. What problem does the code solve that existing packagefeed code does not? Can we rather extend the already existing code, its tests, and its documentation, to support that use case? Alex On Wed, 19 Jul 2023 at 22:03, Charlie Johnston <charlie.johnston@ni.com> wrote: > > Hello, > > I've been working on a packagefeed.bbclass to allow recipes that define a feed. I have a working prototype which has been tested against poky using all 3 package types. > This is my first time submitting this type of change, so I'm looking for some feedback to cover the gaps in my knowledge. > Are there tests that need to be added for this type of change? > Should this type of recipe class be SSTATE cache enabled? > What documentation changes are necessary for this type of change? > > Thanks, > Charlie Johnston > charlie.johnston@ni.com > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#184601): https://lists.openembedded.org/g/openembedded-core/message/184601 > Mute This Topic: https://lists.openembedded.org/mt/100243054/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Specifically please take a look at, ./recipes-core/meta/package-index.bb and generate_index_files() in lib/oe/package_manager/__init__.py I think we should rather extend these instead of adding overlapping code which works differently. Alex On Wed, 19 Jul 2023 at 22:39, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > You need to start by describing the use case. What problem does the > code solve that existing packagefeed code does not? Can we rather > extend the already existing code, its tests, and its documentation, to > support that use case? > > Alex > > > On Wed, 19 Jul 2023 at 22:03, Charlie Johnston <charlie.johnston@ni.com> wrote: > > > > Hello, > > > > I've been working on a packagefeed.bbclass to allow recipes that define a feed. I have a working prototype which has been tested against poky using all 3 package types. > > This is my first time submitting this type of change, so I'm looking for some feedback to cover the gaps in my knowledge. > > Are there tests that need to be added for this type of change? > > Should this type of recipe class be SSTATE cache enabled? > > What documentation changes are necessary for this type of change? > > > > Thanks, > > Charlie Johnston > > charlie.johnston@ni.com > > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#184611): https://lists.openembedded.org/g/openembedded-core/message/184611 > Mute This Topic: https://lists.openembedded.org/mt/100243054/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Sure! Currently to create a feed, users build the packages they want in the feed and then bitbake package-index. That workflow unfortunately has a number of limitations, the mains ones being: - The index creation methods ONLY work on the package deploy directory. If there are packages that are not meant to be in the feed in the deploy directory, they will be included in the package index. Additionally, multiple feeds cannot be built in one command due to this limitation. - If a package feed depends on another package feed being side-by-side to it (that is, if packages in Feed A depend on packages in Feed B and users of Feed B are required to use Feed A) a user would have to manually remove duplicate packages from the deploy directory before making Feed B's index. These changes instead create a packagefeed class that can be inherited and will create a feed in the tmp/deploy/feeds/<feedname> directory. This allows control over the packages to be included in the feed, as the packages are hardlinked or copied from the deploy directory similar to how feeds are constructed for rootfs creation. I could see reusing the logic from the package_manager, but it would require changes to allow indices to be created for arbitrary directories (instead of being hardcoded based on package type), and adding some mechanism to create_packages_dir in the package_manager/__init__.py to handle feed creation tasks/feeds being dependent on other feeds. Are there limitations on changing the API of those classes and methods to add inputs? Alternatively, I could split the overlapping code into a helper method instead? Thanks, Charlie Johnston On 7/19/23 15:59, Alexander Kanavin wrote: > Specifically please take a look at, > ./recipes-core/meta/package-index.bb > and > generate_index_files() in lib/oe/package_manager/__init__.py > > I think we should rather extend these instead of adding overlapping > code which works differently. > > Alex > > On Wed, 19 Jul 2023 at 22:39, Alexander Kanavin via > lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> > wrote: >> >> You need to start by describing the use case. What problem does the >> code solve that existing packagefeed code does not? Can we rather >> extend the already existing code, its tests, and its documentation, to >> support that use case? >> >> Alex >> >> >> On Wed, 19 Jul 2023 at 22:03, Charlie Johnston <charlie.johnston@ni.com> wrote: >>> >>> Hello, >>> >>> I've been working on a packagefeed.bbclass to allow recipes that define a feed. I have a working prototype which has been tested against poky using all 3 package types. >>> This is my first time submitting this type of change, so I'm looking for some feedback to cover the gaps in my knowledge. >>> Are there tests that need to be added for this type of change? >>> Should this type of recipe class be SSTATE cache enabled? >>> What documentation changes are necessary for this type of change? >>> >>> Thanks, >>> Charlie Johnston >>> charlie.johnston@ni.com >>> >>> >>> >>> >>> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#184611): https://lists.openembedded.org/g/openembedded-core/message/184611 >> Mute This Topic: https://lists.openembedded.org/mt/100243054/1686489 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >>
On Wed, 19 Jul 2023 at 23:14, Charlie Johnston <charlie.johnston@ni.com> wrote: > I could see reusing the logic from the package_manager, but it would require > changes to allow indices to be created for arbitrary directories (instead of being > hardcoded based on package type), and adding some mechanism to create_packages_dir > in the package_manager/__init__.py to handle feed creation tasks/feeds being > dependent on other feeds. > > Are there limitations on changing the API of those classes and methods to add > inputs? Alternatively, I could split the overlapping code into a helper method > instead? Thanks for the explanation! The classes were written for the purpose of generating root filesystems for images by constructing filtered package feeds in images' ${WORKDIR}, but I think they should indeed be extended and tweaked to allow generic feeds that can be then exported to deploydirs. In particular create_packages_dir already does what is needed for a filtered feed, and there really shouldn't be a second implementation of that logic elsewhere. Alex