wic/plugins: Source that support both EFI and BIOS

Submitted by William Bourque on July 2, 2019, 6:24 p.m. | Patch ID: 162746

Details

Message ID 20190702182403.24099-1-wbourque@gmail.com
State New
Headers show

Commit Message

William Bourque July 2, 2019, 6:24 p.m.
From: William Bourque <wbourque@gmail.com>

Add a source plugin that support both EFI and legacy PC-Bios.
While using this plugin, both bootloaders configurations reside
in the same /boot partitions.
This plugin has very little code : to avoid code duplication,
we simply re-import bootimg-pcbios and bootmg-efi source and
call both their SourcePlugin methods.

Signed-off-by: William Bourque <wbourque@gmail.com>
---
 .../wic/plugins/source/bootimg-biosplusefi.py | 210 ++++++++++++++++++
 1 file changed, 210 insertions(+)
 create mode 100644 scripts/lib/wic/plugins/source/bootimg-biosplusefi.py

Patch hide | download patch | download mbox

diff --git a/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py b/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py
new file mode 100644
index 0000000000..b7bd8a7fe2
--- /dev/null
+++ b/scripts/lib/wic/plugins/source/bootimg-biosplusefi.py
@@ -0,0 +1,210 @@ 
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This implements the 'bootimg-biosplusefi' source plugin class for 'wic'
+#
+# AUTHORS
+# William Bourque <wbourque [at) gmail.com>
+
+import types
+
+from wic.pluginbase import SourcePlugin
+from importlib.machinery import SourceFileLoader
+
+class BootimgBiosPlusEFIPlugin(SourcePlugin):
+    """
+    Create MBR + EFI boot partition
+
+    This plugin creates a boot partition that contains both
+    legacy BIOS and EFI content. It will be able to boot from both.
+    This is useful when managing PC fleet with some older machines
+    without EFI support.
+
+    Note it is possible to create an image that can boot from both
+    legacy BIOS and EFI by defining two partitions : one with arg
+    --source bootimg-efi  and another one with --source bootimg-pcbios.
+    However, this method has the obvious downside that it requires TWO
+    partitions to be created on the storage device.
+    Both partitions will also be marked as "bootable" which does not work on
+    most BIOS, has BIOS often uses the "bootable" flag to determine 
+    what to boot. If you have such a BIOS, you need to manually remove the 
+    "bootable" flag from the EFI partition for the drive to be bootable.
+    Having two partitions also seems to confuse wic : the content of
+    the first partition will be duplicated into the second, even though it
+    will not be used at all.
+
+    Also, unlike "isoimage-isohybrid" that also does BIOS and EFI, this plugin
+    allows you to have more than only a single rootfs partitions and does
+    not turn the rootfs into an initramfs RAM image.
+
+    This plugin is made to put everything into a single /boot partition so it
+    does not have the limitations listed above.
+
+    The plugin is made so it does tries not to reimplement what's already
+    been done in other plugins; as such it imports "bootimg-pcbios"
+    and "bootimg-efi".
+    Plugin "bootimg-pcbios" is used to generate legacy BIOS boot and
+    plugin "bootimg-efi" is used to generate the UEFI boot.
+    Imports are handled with "SourceFileLoader" from importlib as it is
+    otherwise very difficult to import module that has hyphen "-" in their
+    filename.
+    The SourcePlugin() methods used in the plugins (do_install_disk,
+    do_configure_partition, do_prepare_partition) are then called on both,
+    beginning by "bootimg-efi".
+    
+    Plugin options, such as "--sourceparams" can still be passed to a
+    plugin, as long they does not cause issue in the other plugin.
+
+    Example wic configuration:
+    part /boot --source bootimg-biosplusefi --sourceparams="loader=grub-efi"\\
+               --ondisk sda --label os_boot --active --align 1024 --use-uuid
+    """
+
+    name = 'bootimg-biosplusefi'
+
+    __PCBIOS_MODULE_NAME = "bootimg-pcbios"
+    __EFI_MODULE_NAME = "bootimg-efi"
+
+    __imgEFIObj = None
+    __imgBiosObj = None
+
+    @classmethod
+    def __init__(cls):
+        """
+        Constructor (init)
+        """
+
+        # XXX
+        # For some reasons, __init__ constructor is never called.
+        # Something to do with how pluginbase works?
+        cls.__instanciateSubClasses()
+
+    @classmethod
+    def __instanciateSubClasses(cls):
+        """
+
+        """
+
+        # Import bootimg-pcbios (class name "BootimgPcbiosPlugin")
+        modulePath = os.path.join(os.path.dirname(os.path.realpath(__file__)),
+                                  cls.__PCBIOS_MODULE_NAME + ".py")
+        loader = SourceFileLoader(cls.__PCBIOS_MODULE_NAME, modulePath)
+        mod = types.ModuleType(loader.name)
+        loader.exec_module(mod)
+        cls.__imgBiosObj = mod.BootimgPcbiosPlugin()
+
+        # Import bootimg-efi (class name "BootimgEFIPlugin")
+        modulePath = os.path.join(os.path.dirname(os.path.realpath(__file__)),
+                                  cls.__EFI_MODULE_NAME + ".py")
+        loader = SourceFileLoader(cls.__EFI_MODULE_NAME, modulePath)
+        mod = types.ModuleType(loader.name)
+        loader.exec_module(mod)
+        cls.__imgEFIObj = mod.BootimgEFIPlugin()
+
+    @classmethod
+    def do_install_disk(cls, disk, disk_name, creator, workdir, oe_builddir,
+                        bootimg_dir, kernel_dir, native_sysroot):
+        """
+        Called after all partitions have been prepared and assembled into a
+        disk image.
+        """
+
+        if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ):
+            cls.__instanciateSubClasses()
+
+        cls.__imgEFIObj.do_install_disk(
+            disk,
+            disk_name,
+            creator,
+            workdir,
+            oe_builddir,
+            bootimg_dir,
+            kernel_dir,
+            native_sysroot)
+
+        cls.__imgBiosObj.do_install_disk(
+            disk,
+            disk_name,
+            creator,
+            workdir,
+            oe_builddir,
+            bootimg_dir,
+            kernel_dir,
+            native_sysroot)
+
+    @classmethod
+    def do_configure_partition(cls, part, source_params, creator, cr_workdir,
+                               oe_builddir, bootimg_dir, kernel_dir,
+                               native_sysroot):
+        """
+        Called before do_prepare_partition()
+        """
+
+        if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ):
+            cls.__instanciateSubClasses()
+
+        cls.__imgEFIObj.do_configure_partition(
+            part,
+            source_params,
+            creator,
+            cr_workdir,
+            oe_builddir,
+            bootimg_dir,
+            kernel_dir,
+            native_sysroot)
+
+        cls.__imgBiosObj.do_configure_partition(
+            part,
+            source_params,
+            creator,
+            cr_workdir,
+            oe_builddir,
+            bootimg_dir,
+            kernel_dir,
+            native_sysroot)
+
+    @classmethod
+    def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
+                             oe_builddir, bootimg_dir, kernel_dir,
+                             rootfs_dir, native_sysroot):
+        """
+        Called to do the actual content population for a partition i.e. it
+        'prepares' the partition to be incorporated into the image.
+        """
+
+        if ( (not cls.__imgEFIObj) or (not cls.__imgBiosObj) ):
+            cls.__instanciateSubClasses()
+
+        cls.__imgEFIObj.do_prepare_partition(
+            part,
+            source_params,
+            creator,
+            cr_workdir,
+            oe_builddir,
+            bootimg_dir,
+            kernel_dir,
+            rootfs_dir,
+            native_sysroot)
+
+        cls.__imgBiosObj.do_prepare_partition(
+            part,
+            source_params,
+            creator,
+            cr_workdir,
+            oe_builddir,
+            bootimg_dir,
+            kernel_dir,
+            rootfs_dir,
+            native_sysroot)

Comments

William Bourque July 2, 2019, 7:22 p.m.
Hi,

The error "scripts/lib/wic/plugins/source/bootimg-biosplusefi.py does
not exist" is a bit puzzling : this is the new file my patch is
adding.
Is this test failure expected for new files?

I sent this using git-sendmail; should I submit using a different way
(pull request, ...)?

Thanks,


On Tue, Jul 2, 2019 at 2:30 PM Patchwork
<patchwork@patchwork.openembedded.org> wrote:
>
> == Series Details ==
>
> Series: wic/plugins: Source that support both EFI and BIOS
> Revision: 1
> URL   : https://patchwork.openembedded.org/series/18505/
> State : failure
>
> == Summary ==
>
>
> Thank you for submitting this patch series to OpenEmbedded Core. This is
> an automated response. Several tests have been executed on the proposed
> series by patchtest resulting in the following failures:
>
>
>
> * Issue             Errors in your Python code were encountered [test_pylint]
>   Suggested fix    Correct the lines introduced by your patch
>   Output           Please, fix the listed issues:
>                    scripts/lib/wic/plugins/source/bootimg-biosplusefi.py does not exist
>
>
>
> If you believe any of these test results are incorrect, please reply to the
> mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> Otherwise we would appreciate you correcting the issues and submitting a new
> version of the patchset if applicable. Please ensure you add/increment the
> version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> [PATCH v3] -> ...).
>
> ---
> Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
> Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
>
Richard Purdie July 3, 2019, 7:51 a.m.
On Tue, 2019-07-02 at 15:22 -0400, William Bourque wrote:
> Hi,
> 
> The error "scripts/lib/wic/plugins/source/bootimg-biosplusefi.py does
> not exist" is a bit puzzling : this is the new file my patch is
> adding.
> Is this test failure expected for new files?
> 
> I sent this using git-sendmail; should I submit using a different way
> (pull request, ...)?

I think the test is broken so we can ignore that. I've cc'd Sandy so
she might be able to look into this issue if possible as we've seen it
a few times.

What I did wonder is whether we should be adding some tests to wic for
this new plugin though?

The existing tests are in meta/lib/oeqa/selftest/cases/wic.py and can
be run with "oe-selftest -r wic", would it make sense to add something
to this as part of this change?

Cheers,

Richard
changqing.li@windriver.com July 4, 2019, 9:02 a.m.
On 7/3/19 3:51 PM, Richard Purdie wrote:
> On Tue, 2019-07-02 at 15:22 -0400, William Bourque wrote:
>> Hi,
>>
>> The error "scripts/lib/wic/plugins/source/bootimg-biosplusefi.py does
>> not exist" is a bit puzzling : this is the new file my patch is
>> adding.
>> Is this test failure expected for new files?
>>
>> I sent this using git-sendmail; should I submit using a different way
>> (pull request, ...)?
> I think the test is broken so we can ignore that. I've cc'd Sandy so
> she might be able to look into this issue if possible as we've seen it
> a few times.

I checked the test case,  and current test only support modified file well,

for new add file,  enhancement is needed, I have file a bug in bugzilla,

I will fix this when have time.

>
> What I did wonder is whether we should be adding some tests to wic for
> this new plugin though?
>
> The existing tests are in meta/lib/oeqa/selftest/cases/wic.py and can
> be run with "oe-selftest -r wic", would it make sense to add something
> to this as part of this change?
>
> Cheers,
>
> Richard
>
>
changqing.li@windriver.com July 4, 2019, 9:03 a.m.
On 7/3/19 3:51 PM, Richard Purdie wrote:
> On Tue, 2019-07-02 at 15:22 -0400, William Bourque wrote:
>> Hi,
>>
>> The error "scripts/lib/wic/plugins/source/bootimg-biosplusefi.py does
>> not exist" is a bit puzzling : this is the new file my patch is
>> adding.
>> Is this test failure expected for new files?
>>
>> I sent this using git-sendmail; should I submit using a different way
>> (pull request, ...)?
> I think the test is broken so we can ignore that. I've cc'd Sandy so
> she might be able to look into this issue if possible as we've seen it
> a few times.

I checked the test case,  and current test only support modified file well,

for new add file,  enhancement is needed, I have file a bug in bugzilla,

I will fix this when have time.

>
> What I did wonder is whether we should be adding some tests to wic for
> this new plugin though?
>
> The existing tests are in meta/lib/oeqa/selftest/cases/wic.py and can
> be run with "oe-selftest -r wic", would it make sense to add something
> to this as part of this change?
>
> Cheers,
>
> Richard
>
>
William Bourque July 4, 2019, 4:29 p.m.
On Wed, Jul 3, 2019 at 3:51 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2019-07-02 at 15:22 -0400, William Bourque wrote:
> > Hi,
> >
> > The error "scripts/lib/wic/plugins/source/bootimg-biosplusefi.py does
> > not exist" is a bit puzzling : this is the new file my patch is
> > adding.
> > Is this test failure expected for new files?
> >
> > I sent this using git-sendmail; should I submit using a different way
> > (pull request, ...)?
>
> I think the test is broken so we can ignore that. I've cc'd Sandy so
> she might be able to look into this issue if possible as we've seen it
> a few times.
>
> What I did wonder is whether we should be adding some tests to wic for
> this new plugin though?
>
> The existing tests are in meta/lib/oeqa/selftest/cases/wic.py and can
> be run with "oe-selftest -r wic", would it make sense to add something
> to this as part of this change?
>
> Cheers,
>
> Richard
>

You are right, some tests would be a good idea.

That said, I am a bit confused on how to integrate in
meta/lib/oeqa/selftest/cases/wic.py
Most tests in there check generic functionnalities, I'm not quite sure
how to use it to tests the new SourcePlugin() specific
functionnalities, instead of just the generic.
I will come up with something but it might not be great... but "not
great" is better than "nothing" I suppose.

Do you want me to resubmit the SourcePlugin code and the tests together?
Or would you rather have me submit the tests separately?

Thanks

- William