| Message ID | 20240412090234.4110915-1-michael.opdenacker@bootlin.com |
|---|---|
| Headers | show |
| Series | prserv: add support for an "upstream" server | expand |
On Fri, Apr 12, 2024 at 5:02 AM Michael Opdenacker via lists.openembedded.org <michael.opdenacker= bootlin.com@lists.openembedded.org> wrote: > From: Michael Opdenacker <michael.opdenacker@bootlin.com> > > This makes it possible to customize an "upstream" distribution > by modifying local packages. If the "upstream" package bears > revision "x", the local one will have revision "x.y", this > having priority over the upstream one. > > Multiple levels of upstream servers are supported, so "x.y.z" revisions > are possible too. > > Take advantage of this work to clean-up and update the prserv code too. > > Manual "sanity" tests have been run so far but automated tests > (based on Python unittest, as for the hash server tests) > are being prepared for the next iteration. They will also to > test also supported scenarios. > > Code reviews are welcome, especially for more Pythonic > ways of implementing some details. > > I went through the code and commented on a few of the larger patches. I can't say that I'm an expert in the existing PR server and I don't tend to write overly pythonic code .. but I did look through things and can add: Reviewed-by: Bruce Ashfield <bruce.ashfield@gmail.com> > --- > > Changes in V3: > > - Revert the commit removing the so far unused "hist" mode, which > we wish to keep for binary reproducibility sake. > > - Simplification of get_value() function to take > both "hist" and "nohist" modes with the same shared code. > > - Add "history" parameter to the "getPR" request, > so that the client can ask for the mode of its choice. > This will also make it possible to implement tests > for both modes. > > Note that more requests ("export", "import"...) > will also need a "history" parameter, in a future version, > after the first tests are implemented. > > - Several fixes to bugs either present in V2 or introduced > by the V3 code changes. > > - Put all the new features at the tip of the branch, > to make the cleanup commits easier to merged. > > Changes in V2: > > - Add this new commit: > prserv: remove unused "hist" mode in the database backend > > - Squash commit "prserv: fix read_only test" into > commit "prserv: simplify the PRServerClient() interface" > (Reported by Richard Purdie) > > - Fix the code to support increasing "x.y.z" values, thus > supporting several levels of upstream servers. > > - db.py: remove duplicate definition of find_max_value() function in db.py > > - prserv.py: remove tabs before comments (Python didn't complain) > > - db.py: now stores the revision ("value") as TEXT. > This way we can store "1.0" without having it transformed to "1" > when the default type was INTEGER. > > - This allows to fix a regression when the first packages were created > with 'r0.1' instead of 'r0.0' initially. > > - find_max_value: now returns None instead of '0' when no value is found > Before we couldn't tell the difference between a '0' > max value and the absence of such a value. > > Michael Opdenacker (12): > prserv: simplify the PRServerClient() interface > prserv: use double quotes by default > bitbake-prserv: replace deprecated optparse by argparse > prserv: use self.logger instead of logger directly > asyncrpc: include parse_address from hashserv > prserv: capitalization and spacing improvements > prserv: add extra requests > prserv: remove redundant exception handler > prserv: correct error message > prserv: remove unnecessary code > prserv: add "upstream" server support > prserv: add "history" argument to "get-pr" request > > bin/bitbake-prserv | 99 ++++++++++++----- > lib/bb/asyncrpc/client.py | 23 ++++ > lib/hashserv/__init__.py | 27 +---- > lib/prserv/__init__.py | 21 +++- > lib/prserv/client.py | 44 ++++++-- > lib/prserv/db.py | 225 ++++++++++++++++++++++---------------- > lib/prserv/serv.py | 220 ++++++++++++++++++++++++++----------- > 7 files changed, 436 insertions(+), 223 deletions(-) > > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#16075): > https://lists.openembedded.org/g/bitbake-devel/message/16075 > Mute This Topic: https://lists.openembedded.org/mt/105479088/1050810 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ > bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
From: Michael Opdenacker <michael.opdenacker@bootlin.com> This makes it possible to customize an "upstream" distribution by modifying local packages. If the "upstream" package bears revision "x", the local one will have revision "x.y", this having priority over the upstream one. Multiple levels of upstream servers are supported, so "x.y.z" revisions are possible too. Take advantage of this work to clean-up and update the prserv code too. Manual "sanity" tests have been run so far but automated tests (based on Python unittest, as for the hash server tests) are being prepared for the next iteration. They will also to test also supported scenarios. Code reviews are welcome, especially for more Pythonic ways of implementing some details. --- Changes in V3: - Revert the commit removing the so far unused "hist" mode, which we wish to keep for binary reproducibility sake. - Simplification of get_value() function to take both "hist" and "nohist" modes with the same shared code. - Add "history" parameter to the "getPR" request, so that the client can ask for the mode of its choice. This will also make it possible to implement tests for both modes. Note that more requests ("export", "import"...) will also need a "history" parameter, in a future version, after the first tests are implemented. - Several fixes to bugs either present in V2 or introduced by the V3 code changes. - Put all the new features at the tip of the branch, to make the cleanup commits easier to merged. Changes in V2: - Add this new commit: prserv: remove unused "hist" mode in the database backend - Squash commit "prserv: fix read_only test" into commit "prserv: simplify the PRServerClient() interface" (Reported by Richard Purdie) - Fix the code to support increasing "x.y.z" values, thus supporting several levels of upstream servers. - db.py: remove duplicate definition of find_max_value() function in db.py - prserv.py: remove tabs before comments (Python didn't complain) - db.py: now stores the revision ("value") as TEXT. This way we can store "1.0" without having it transformed to "1" when the default type was INTEGER. - This allows to fix a regression when the first packages were created with 'r0.1' instead of 'r0.0' initially. - find_max_value: now returns None instead of '0' when no value is found Before we couldn't tell the difference between a '0' max value and the absence of such a value. Michael Opdenacker (12): prserv: simplify the PRServerClient() interface prserv: use double quotes by default bitbake-prserv: replace deprecated optparse by argparse prserv: use self.logger instead of logger directly asyncrpc: include parse_address from hashserv prserv: capitalization and spacing improvements prserv: add extra requests prserv: remove redundant exception handler prserv: correct error message prserv: remove unnecessary code prserv: add "upstream" server support prserv: add "history" argument to "get-pr" request bin/bitbake-prserv | 99 ++++++++++++----- lib/bb/asyncrpc/client.py | 23 ++++ lib/hashserv/__init__.py | 27 +---- lib/prserv/__init__.py | 21 +++- lib/prserv/client.py | 44 ++++++-- lib/prserv/db.py | 225 ++++++++++++++++++++++---------------- lib/prserv/serv.py | 220 ++++++++++++++++++++++++++----------- 7 files changed, 436 insertions(+), 223 deletions(-)