[bitbake-devel] ipk ar/tar unpack

Submitted by Chris Minnoy on July 15, 2020, 1:20 p.m. | Patch ID: 174449

Details

Message ID a3910c7d-89b4-6203-7ff8-339205e249e4@softathome.com
State New
Headers show

Commit Message

Chris Minnoy July 15, 2020, 1:20 p.m.
From: Chris Minnoy <chris.minnoy_ext@softathome.com>
Date: Wed, 15 Jul 2020 15:02:35 +0200
Subject: [PATCH] ipk packages can be formed with ar or tar. ar/tar can't 
read
  each other archives. This fixes the unpack to call the correct 
archiver based
  on the format.

---
  bitbake/lib/bb/fetch2/__init__.py | 40 ++++++++++++++++++++++++++++---
  1 file changed, 37 insertions(+), 3 deletions(-)

-xpf %s && rm %s' % (file, datafile, datafile, datafile)
+
+                    else:
+                        raise UnpackError("Unable to unpack ipk package 
- unknown format", urldata.url)
+                else:
+                    raise UnpackError("Unable to detect .ipk file 
format - unknown format", urldata.url)
+            elif file.endswith('.deb'):
                  output = subprocess.check_output(['ar', '-t', file], 
preexec_fn=subprocess_setup)
                  datafile = None
                  if output:
@@ -1476,9 +1510,9 @@ class FetchMethod(object):
                              datafile = line
                              break
                      else:
-                        raise UnpackError("Unable to unpack deb/ipk 
package - does not contain data.tar.* file", urldata.url)
+                        raise UnpackError("Unable to unpack deb package 
- does not contain data.tar.* file", urldata.url)
                  else:
-                    raise UnpackError("Unable to unpack deb/ipk package 
- could not list contents", urldata.url)
+                    raise UnpackError("Unable to unpack deb package - 
could not list contents", urldata.url)
                  cmd = 'ar x %s %s && tar --no-same-owner -xpf %s && rm 
%s' % (file, datafile, datafile, datafile)

          # If 'subdir' param exists, create a dir and use it as 
destination for unpack cmd

Patch hide | download patch | download mbox

diff --git a/bitbake/lib/bb/fetch2/__init__.py 
b/bitbake/lib/bb/fetch2/__init__.py
index 2ada43befc..e79149a208 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -1467,7 +1467,41 @@  class FetchMethod(object):
                      iterate_file = unpack_file
                  else:
                      cmd = 'rpm2cpio.sh %s | cpio -id' % (file)
-            elif file.endswith('.deb') or file.endswith('.ipk'):
+            elif file.endswith('.ipk'):
+                output = subprocess.check_output(['file', file], 
preexec_fn=subprocess_setup)
+                if output:
+                    if output.decode().find("current ar archive") != -1:
+                        output = subprocess.check_output(['ar', '-t', 
file], preexec_fn=subprocess_setup)
+                        datafile = None
+                        if output:
+                            for line in output.decode().splitlines():
+                                if line.startswith('data.tar.'):
+                                    datafile = line
+                                    break
+                            else:
+                                raise UnpackError("Unable to unpack ipk 
package - does not contain data.tar.* file", urldata.url)
+                        else:
+                            raise UnpackError("Unable to unpack ipk 
package - could not list contents", urldata.url)
+                        cmd = 'ar x %s %s && tar --no-same-owner -xpf 
%s && rm %s' % (file, datafile, datafile, datafile)
+                    elif output.decode().find("gzip compressed data") 
!= -1:
+                        output = subprocess.check_output(['tar', '-tf', 
file], preexec_fn=subprocess_setup)
+                        datafile = None
+                        if output:
+                            for line in output.decode().splitlines():
+                                if line.startswith('./data.tar.'):
+                                    datafile = line
+                                    break
+                            else:
+                                raise UnpackError("Unable to unpack ipk 
package - does not contain data.tar.* file", urldata.url)
+                        else:
+                            raise UnpackError("Unable to unpack ipk 
package - could not list contents", urldata.url)
+                        cmd = 'tar -xf %s %s && tar --no-same-owner 

Comments

Richard Purdie July 19, 2020, 3:25 p.m.
Hi,

On Wed, 2020-07-15 at 15:20 +0200, Chris Minnoy wrote:
> From: Chris Minnoy <chris.minnoy_ext@softathome.com>
> Date: Wed, 15 Jul 2020 15:02:35 +0200
> Subject: [PATCH] ipk packages can be formed with ar or tar. ar/tar
> can't 
> read
>   each other archives. This fixes the unpack to call the correct 
> archiver based
>   on the format.
> 
> ---
>   bitbake/lib/bb/fetch2/__init__.py | 40
> ++++++++++++++++++++++++++++---
>   1 file changed, 37 insertions(+), 3 deletions(-)
> 
> diff --git a/bitbake/lib/bb/fetch2/__init__.py 
> b/bitbake/lib/bb/fetch2/__init__.py
> index 2ada43befc..e79149a208 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -1467,7 +1467,41 @@ class FetchMethod(object):
>                       iterate_file = unpack_file
>                   else:
>                       cmd = 'rpm2cpio.sh %s | cpio -id' % (file)
> -            elif file.endswith('.deb') or file.endswith('.ipk'):
> +            elif file.endswith('.ipk'):
> +                output = subprocess.check_output(['file', file], 
> preexec_fn=subprocess_setup)
> +                if output:

Thanks for the patch. Unfortunately it came through line wrapped. The
short/long commit message also doesn't match our patch format
guidelines.

https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines

Could you fix those issues and resubmit please?

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11503): https://lists.openembedded.org/g/bitbake-devel/message/11503
Mute This Topic: https://lists.openembedded.org/mt/75519726/3617530
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub  [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-
Chris Minnoy Aug. 7, 2020, 11:54 a.m.
From d21155c4b55ac96d3c13a3cef1f020df67b62641 Mon Sep 17 00:00:00 2001
From: Chris Minnoy <chris.minnoy_ext@softathome.com>
Date: Fri, 7 Aug 2020 11:35:27 +0000
Subject: [PATCH 1/1] bitbake fetch2: fetcher can now unpack .ipk files from ar and tar format

Ipk packages can be formed with ar or tar. ar/tar can't read each others archives.
This fixes the unpack to call the correct archiver based on the format.
---
bitbake/lib/bb/fetch2/__init__.py | 39 ++++++++++++++++++++++++++++++++++++---
1 file changed, 36 insertions(+), 3 deletions(-)

--
2.11.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11549): https://lists.openembedded.org/g/bitbake-devel/message/11549
Mute This Topic: https://lists.openembedded.org/mt/75519726/3617530
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub  [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-
Richard Purdie Aug. 7, 2020, 1:01 p.m.
On Fri, 2020-08-07 at 04:54 -0700, Chris Minnoy wrote:
> From d21155c4b55ac96d3c13a3cef1f020df67b62641 Mon Sep 17 00:00:00
> 2001
> From: Chris Minnoy <chris.minnoy_ext@softathome.com>
> Date: Fri, 7 Aug 2020 11:35:27 +0000
> Subject: [PATCH 1/1] bitbake fetch2: fetcher can now unpack .ipk
> files from ar and tar format
> 
> Ipk packages can be formed with ar or tar. ar/tar can't read each
> others archives.
> This fixes the unpack to call the correct archiver based on the
> format.

What is creating ipk files using tar instead of ar?

I know they did once exist but I've not seen real world use of those
for many years?

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11552): https://lists.openembedded.org/g/bitbake-devel/message/11552
Mute This Topic: https://lists.openembedded.org/mt/75519726/3617530
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub  [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-