[RFC,v2,1/2] bitbake-layers: Add ability to update the reference of repositories

From: Jermain Horsman <jermain.horsman@nedap.com>

This uses an existing setup-layers configuration and modifies one
or more repositories using a reference provided by the user.

This is a very minimal implementation, no validation of this reference
is done and it is left to the user to provide a valid value.

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
---
 meta/lib/bblayers/makesetup.py                | 11 +++++
 .../bblayers/setupwriters/oe-setup-layers.py  | 43 ++++++++++++++++---
 2 files changed, 47 insertions(+), 7 deletions(-)

On Thu, 4 Jan 2024 at 13:35, Jermain Horsman <jermain.horsman@nedap.com> wrote:
> +        parser_setup_layers.add_argument('--update', '-u',
> +            action='append',
> +            metavar='REPOSITORY',
> +            help='Repository to update, this requires a reference to be specified.\nThis option can be used multiple times.')
> +        parser_setup_layers.add_argument('--reference', '-r',
> +            help="Reference to use when updating repositories.\nThe value can be any git reference, however it is up to the user to provide a valid value.\nThis option is only useful when using '--update'.")
> +
>          self.plugins = []

These options should move to the plugin. If/when we have another
plugin, we can consider making them global, until then we can't assume
all plugins would want to handle these.

>      def do_write(self, parent, args):
>          """ Writes out a python script and a json config that replicate the directory structure and revisions of the layers in a current build. """
> -        if not os.path.exists(args.destdir):
> -            os.makedirs(args.destdir)
> -        repos = parent.make_repo_config(args.destdir)
> -        json = {"version":"1.0","sources":repos}
> -        if not repos:
> -            raise Exception("Could not determine layer sources")
>          output = args.output_prefix or "setup-layers"
> -        output = os.path.join(os.path.abspath(args.destdir),output)
> +        output = os.path.join(os.path.abspath(args.destdir), output)
> +
> +        if args.update is None:
> +            if not os.path.exists(args.destdir):
> +                os.makedirs(args.destdir)
> +            repos = parent.make_repo_config(args.destdir)
> +            json = {"version":"1.0","sources":repos}
> +            if not repos:
> +                raise Exception("Could not determine layer sources")
> +        else:
> +            json = self._read_repo_config(output + ".json")
> +            if not 'sources' in json.keys():
> +                raise Exception("File {}.json does not contain valid layer sources.".format(output))
> +            self._modify_repo_config(json, args)
> +
>          self._write_json(json, output + ".json")
>          logger.info('Created {}.json'.format(output))
>          if not args.json_only:

--update requires that the json file already exists. But why should
it? It's entirely possible to write a new json, but update the
references from command line arguments just before writing the file
out. No?

I also wonder if we can name these two options in a better way. Maybe
'--use-custom-reference repo', instead of --update?

Alex

On Thu, 4 Jan 2024 at 19:07, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:

> --update requires that the json file already exists. But why should
> it? It's entirely possible to write a new json, but update the
> references from command line arguments just before writing the file
> out. No?
>
> I also wonder if we can name these two options in a better way. Maybe
> '--use-custom-reference repo', instead of --update?

On further thought, if done this way, then --update could instead
serve to update file that already exists, whereas the default would be
to overwrite the file with a new one.

Alex

> On further thought, if done this way, then --update could instead
> serve to update file that already exists, whereas the default would be

I would prefer this, as I might have a config with repo A and B for which
I have changed both locally, but I'd only want to apply the changes to B
to the layers setup, writing a new file will update A, whereas using the
old layers setup will leave A as is.

> to overwrite the file with a new one.

To clarify as it is not entirely clear this is what you suggested.

Use '--update' as a boolean flag which determines whether or not to use
the old layers setup or write a new one
and use an additional '--use-custom-reference repo' option to determine
for with repo to use the custom reference.

This leave the question of what to do if only the '--reference' option is used?
Do nothing or update/use a custom revision for all repos?
An separate option like '--use-custom-reference-all' seems undesirable to me.

Sincerely,

Jermain Horsman

On Fri, 5 Jan 2024 at 12:14, Jermain Horsman <jermain.horsman@nedap.com> wrote:
> This leave the question of what to do if only the '--reference' option is used?
> Do nothing or update/use a custom revision for all repos?

Updating all repos to the same reference is unlikely to work well with
real world combinations of layer repositories. I would do nothing and
print a message indicating so.

Alex