[kirkstone,24/24] npm: use npm_registry to cache package

From: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>

With nodejs 16, the simple 'npm cache add' approach does not work
anymore because its fetcher implementation downloads also meta
information from the registry.

We have to generate these information and add them to the cache.
There is no direct support in 'npm' for task so we have to implement
it manually.

This implementation consists of a openembedded python module (in
oe-core) and a nodejs version specific helper (in oe-meta).

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 019b9c341d539939098962c228c1fd5c99331312)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes/npm.bbclass | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

FWIW: this depends on meta-oe change to add meta-oe-cache-native as
reported in:
https://github.com/openembedded/meta-openembedded/issues/606
I've sent the backport request for meta-oe/kirkstone:
https://lists.openembedded.org/g/openembedded-devel/message/98787

On Wed, Sep 7, 2022 at 4:21 PM Steve Sakoman <steve@sakoman.com> wrote:

> From: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
>
> With nodejs 16, the simple 'npm cache add' approach does not work
> anymore because its fetcher implementation downloads also meta
> information from the registry.
>
> We have to generate these information and add them to the cache.
> There is no direct support in 'npm' for task so we have to implement
> it manually.
>
> This implementation consists of a openembedded python module (in
> oe-core) and a nodejs version specific helper (in oe-meta).
>
> Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> (cherry picked from commit 019b9c341d539939098962c228c1fd5c99331312)
> Signed-off-by: Steve Sakoman <steve@sakoman.com>
> ---
>  meta/classes/npm.bbclass | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
> index 11c80a738e..8379c7b988 100644
> --- a/meta/classes/npm.bbclass
> +++ b/meta/classes/npm.bbclass
> @@ -19,7 +19,7 @@
>
>  inherit python3native
>
> -DEPENDS:prepend = "nodejs-native "
> +DEPENDS:prepend = "nodejs-native nodejs-oe-cache-native "
>  RDEPENDS:${PN}:append:class-target = " nodejs"
>
>  EXTRA_OENPM = ""
> @@ -46,6 +46,7 @@ NPM_ARCH ?=
> "${@npm_target_arch_map(d.getVar("TARGET_ARCH"))}"
>  NPM_PACKAGE = "${WORKDIR}/npm-package"
>  NPM_CACHE = "${WORKDIR}/npm-cache"
>  NPM_BUILD = "${WORKDIR}/npm-build"
> +NPM_REGISTRY = "${WORKDIR}/npm-registry"
>
>  def npm_global_configs(d):
>      """Get the npm global configuration"""
> @@ -109,16 +110,18 @@ python npm_do_configure() {
>      from bb.fetch2.npm import npm_unpack
>      from bb.fetch2.npmsw import foreach_dependencies
>      from bb.progress import OutOfProgressHandler
> +    from oe.npm_registry import NpmRegistry
>
>      bb.utils.remove(d.getVar("NPM_CACHE"), recurse=True)
>      bb.utils.remove(d.getVar("NPM_PACKAGE"), recurse=True)
>
>      env = NpmEnvironment(d, configs=npm_global_configs(d))
> +    registry = NpmRegistry(d.getVar('NPM_REGISTRY'),
> d.getVar('NPM_CACHE'))
>
> -    def _npm_cache_add(tarball):
> -        """Run 'npm cache add' for a specified tarball"""
> -        cmd = "npm cache add %s" % shlex.quote(tarball)
> -        env.run(cmd)
> +    def _npm_cache_add(tarball, pkg):
> +        """Add tarball to local registry and register it in the
> +           cache"""
> +        registry.add_pkg(tarball, pkg)
>
>      def _npm_integrity(tarball):
>          """Return the npm integrity of a specified tarball"""
> @@ -182,7 +185,7 @@ python npm_do_configure() {
>              # Add the dependency to the npm cache
>              destdir = os.path.join(d.getVar("S"), destsuffix)
>              (tarball, pkg) = npm_pack(env, destdir, tmpdir)
> -            _npm_cache_add(tarball)
> +            _npm_cache_add(tarball, pkg)
>              # Add its signature to the cached shrinkwrap
>              dep = _npmsw_dependency_dict(cached_shrinkwrap, deptree)
>              dep["version"] = pkg['version']
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#170419):
> https://lists.openembedded.org/g/openembedded-core/message/170419
> Mute This Topic: https://lists.openembedded.org/mt/93525927/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>