lib/oe/package: fix LOCALE_PATHS scan to create locale packages

split_locales() must only check subdirectories in paths added to
LOCALE_PATHS to avoid creating weird packages based on filenames also
present in paths.

Without such a filter, cups recipe adding ${datadir}/cups/templates to
LOCALE_PATHS creates the following incorrect packages:
- cups-locale-add-class.tmpl
- cups-locale-add-printer.tmpl
- cups-locale-admin.tmpl

Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
---
 meta/conf/documentation.conf |  2 +-
 meta/lib/oe/package.py       | 14 +++++++++-----
 2 files changed, 10 insertions(+), 6 deletions(-)


base-commit: f5e6183b9557477bef74024a587de0bfcc2b7c0d

On Mon, 2024-02-19 at 16:19 +0000, Jonathan GUILLOT wrote:
> split_locales() must only check subdirectories in paths added to
> LOCALE_PATHS to avoid creating weird packages based on filenames also
> present in paths.
> 
> Without such a filter, cups recipe adding ${datadir}/cups/templates
> to
> LOCALE_PATHS creates the following incorrect packages:
> - cups-locale-add-class.tmpl
> - cups-locale-add-printer.tmpl
> - cups-locale-admin.tmpl
> 
> Signed-off-by: Jonathan GUILLOT <jonathan@joggee.fr>
> ---
>  meta/conf/documentation.conf |  2 +-
>  meta/lib/oe/package.py       | 14 +++++++++-----
>  2 files changed, 10 insertions(+), 6 deletions(-)
> 
> diff --git a/meta/conf/documentation.conf
> b/meta/conf/documentation.conf
> index 90d8e82932..b0591881ba 100644
> --- a/meta/conf/documentation.conf
> +++ b/meta/conf/documentation.conf
> @@ -271,7 +271,7 @@ LICENSE_PATH[doc] = "Path to additional licenses
> used during the build."
>  LINUX_KERNEL_TYPE[doc] = "Defines the kernel type to be used in
> assembling the configuration."
>  LINUX_VERSION[doc] = "The Linux version from kernel.org on which the
> Linux kernel image being built using the OpenEmbedded build system is
> based. You define this variable in the kernel recipe."
>  LINUX_VERSION_EXTENSION[doc] = "A string extension compiled into the
> version string of the Linux kernel built with the OpenEmbedded build
> system. You define this variable in the kernel recipe."
> -LOCALE_PATHS[doc] = "Whitespace separated list of paths that are
> scanned to construct locale packages. The list already contains
> ${datadir}/locale by default."
> +LOCALE_PATHS[doc] = "Whitespace separated list of paths that are
> scanned to construct locale packages. The list already contains
> ${datadir}/locale by default. Note that all subdirectories in these
> paths are assumed to be locales."
>  LOCALE_UTF8_IS_DEFAULT[doc] = "If set, locale names are renamed such
> that those lacking an explicit encoding (e.g. en_US) will always be
> UTF-8, and non-UTF-8 encodings are renamed to, e.g., en_US.ISO-8859-
> 1. Otherwise, the encoding is specified by glibc's SUPPORTED file.
> Not supported for precompiled locales."
>  LOG_DIR[doc] = "Specifies the directory to which the OpenEmbedded
> build system writes overall log files. The default directory is
> ${TMPDIR}/log"
>  
> diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
> index d1738d3b61..587810bdaf 100644
> --- a/meta/lib/oe/package.py
> +++ b/meta/lib/oe/package.py
> @@ -652,11 +652,15 @@ def split_locales(d):
>      locales = set()
>      for localepath in (d.getVar('LOCALE_PATHS') or "").split():
>          localedir = dvar + localepath
> -        if cpath.isdir(localedir):
> -            locales.update(os.listdir(localedir))
> -            localepaths.append(localepath)
> -        else:
> -            bb.debug(1, "No locale files in %s" % localepath)
> +        if not cpath.isdir(localedir):
> +            bb.debug(1, 'No locale files in %s' % localepath)
> +            continue
> +
> +        localepaths.append(localepath)
> +        with os.scandir(localedir) as it:
> +            for entry in it:
> +                if entry.is_dir():
> +                    locales.add(entry.name)
>  
>      if len(locales) == 0:
>          bb.debug(1, "No locale files in this package")
> 
> base-commit: f5e6183b9557477bef74024a587de0bfcc2b7c0d

Unfortunately this breaks glibc-locale:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6379/steps/14/logs/stdio

ERROR: glibc-locale-2.39+git-r0 do_package: QA Issue: glibc-locale: Files/directories were installed but not shipped in any package:
  /usr/share/locale/locale.alias
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.

Cheers,

Richard

Without any of my modifications, it seems that glibc-locale recipe creates
a glibc-locale-locale.alias package containing only the file
/usr/share/locale/locale.alias. It looks like the same issue revealed with
the CUPS recipe. Do you know in which package locale.alias file is expected
to be located?

Regards,
Jonathan

Le mar. 20 févr. 2024 à 08:42, Richard Purdie <
richard.purdie@linuxfoundation.org> a écrit :

> ERROR: glibc-locale-2.39+git-r0 do_package: QA Issue: glibc-locale:
> Files/directories were installed but not shipped in any package:
>   /usr/share/locale/locale.alias
> Please set FILES such that these items are packaged. Alternatively if they
> are unneeded, avoid installing them or delete them within do_install.
>

I have attached a patch (as an example) recreating the same
glibc-locale-locale.alias package but it is really weird to me. Is there a
glibc locale expert to help?

Regards,
Jonathan

Le mar. 20 févr. 2024 à 10:45, Jonathan GUILLOT <jonathan@joggee.fr> a
écrit :

> Without any of my modifications, it seems that glibc-locale recipe creates
> a glibc-locale-locale.alias package containing only the file
> /usr/share/locale/locale.alias. It looks like the same issue revealed with
> the CUPS recipe. Do you know in which package locale.alias file is expected
> to be located?
>
> Regards,
> Jonathan
>
> Le mar. 20 févr. 2024 à 08:42, Richard Purdie <
> richard.purdie@linuxfoundation.org> a écrit :
>
>> ERROR: glibc-locale-2.39+git-r0 do_package: QA Issue: glibc-locale:
>> Files/directories were installed but not shipped in any package:
>>   /usr/share/locale/locale.alias
>> Please set FILES such that these items are packaged. Alternatively if
>> they are unneeded, avoid installing them or delete them within do_install.
>>
>

On Thu, 2024-02-22 at 11:56 +0100, Jonathan GUILLOT wrote:
> I have attached a patch (as an example) recreating the same glibc-
> locale-locale.alias package but it is really weird to me. Is there a
> glibc locale expert to help?

I had a look at the situation and I think the best thing to do is
package this into a "glibc-locale-alias" package. I think the file
itself is obsolete and not really used so moving to a more logically
named package just in case anyone needs it is probably the bets way
forward.

Cheers,

Richard