Patchwork [RFC,4/4] lib/oeqa: add a test target controller for EFI targets

login
register
mail settings
Submitter Stanacar, StefanX
Date March 20, 2014, 4:29 p.m.
Message ID <726442aad46caaed6a069b6428692ed2e4903026.1395332218.git.stefanx.stanacar@intel.com>
Download mbox | patch
Permalink /patch/68955/
State New
Headers show

Comments

Stanacar, StefanX - March 20, 2014, 4:29 p.m.
The purpose of this module is to deploy a test image on a EFI-enabled hardware
and run our runtime tests.
A bit of background:
 - testimage.bbclass uses the concept of TEST_TARGET which is a class name
that is responsible for target deploying. A layer can provide
it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget
(ssh into an already up and running machine and run tests), the default one being qemu.
 - basically testimage does something like:
      - target.deploy()
      - target.start()
      - runTests()
      - target.stop()

This module assumes a running EFI (and gummiboot as bootloader) machine with
core-image-testmaster installed (or similar).
In order to use this Master Image mechanism there are some hard requirements:
 - it only works for EFI-enabled hardware with EFI_PROVIDER = "gummiboot"
 - your hardware under test has to be in DHCP-enabled network that gives it the same IP for each reboot

One time setup (master image):
 - build core-image-testmaster with EFI_PROVIDER = "gummiboot"
 - TEMP: for now you'll need SRCREV_meta_pn-linux-yocto = "${AUTOREV}"
    in local.conf so that some kernel configs required by gummiboot get enabled
 - install the image on the target

Test image setup:
 - build your test image, e.g core-image-sato as you usually do, but with these in local.conf:
    IMAGE_FSTYPES += "tar.gz"
   (similar to above you need SRCREV_meta_pn-linux-yocto = "${AUTOREV}")
 - Now run the tests:
    INHERIT += "testimage"
    TEST_TARGET = "GenericEfi"
    TEST_TARGET_IP = "192.168.2.3"
    bitbake core-image-sato -c testimage

Other notes:
 - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling.
   The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like
   TEST_POWERCONTROL_CMD="/home/stefans/powercontrol.exp test 192.168.2.1 nuc1" in local.conf.
   Basically my expect script does: 'ssh test@192.168.2.1 and runs a python script there that controls power for a label called nuc1'.
 - if no command is defined it would use classic reboot. This is fine as long as the machine
   actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where
   some manual interaction is okay from time to time.

[YOCTO #5614]

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/controllers/masterimage.py | 124 +++++++++++++++++++++++++++++++
 meta/lib/oeqa/runtime/ssh.py             |   2 +
 2 files changed, 126 insertions(+)
 create mode 100644 meta/lib/oeqa/controllers/masterimage.py
Otavio Salvador - March 20, 2014, 9:08 p.m.
On Thu, Mar 20, 2014 at 1:29 PM, Stefan Stanacar
<stefanx.stanacar@intel.com> wrote:
> The purpose of this module is to deploy a test image on a EFI-enabled hardware
> and run our runtime tests.
> A bit of background:
>  - testimage.bbclass uses the concept of TEST_TARGET which is a class name
> that is responsible for target deploying. A layer can provide
> it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget
> (ssh into an already up and running machine and run tests), the default one being qemu.
>  - basically testimage does something like:
>       - target.deploy()
>       - target.start()
>       - runTests()
>       - target.stop()
>
> This module assumes a running EFI (and gummiboot as bootloader) machine with
> core-image-testmaster installed (or similar).
> In order to use this Master Image mechanism there are some hard requirements:
>  - it only works for EFI-enabled hardware with EFI_PROVIDER = "gummiboot"
>  - your hardware under test has to be in DHCP-enabled network that gives it the same IP for each reboot
>
> One time setup (master image):
>  - build core-image-testmaster with EFI_PROVIDER = "gummiboot"
>  - TEMP: for now you'll need SRCREV_meta_pn-linux-yocto = "${AUTOREV}"
>     in local.conf so that some kernel configs required by gummiboot get enabled
>  - install the image on the target
>
> Test image setup:
>  - build your test image, e.g core-image-sato as you usually do, but with these in local.conf:
>     IMAGE_FSTYPES += "tar.gz"
>    (similar to above you need SRCREV_meta_pn-linux-yocto = "${AUTOREV}")
>  - Now run the tests:
>     INHERIT += "testimage"
>     TEST_TARGET = "GenericEfi"
>     TEST_TARGET_IP = "192.168.2.3"
>     bitbake core-image-sato -c testimage
>
> Other notes:
>  - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling.
>    The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like
>    TEST_POWERCONTROL_CMD="/home/stefans/powercontrol.exp test 192.168.2.1 nuc1" in local.conf.
>    Basically my expect script does: 'ssh test@192.168.2.1 and runs a python script there that controls power for a label called nuc1'.
>  - if no command is defined it would use classic reboot. This is fine as long as the machine
>    actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where
>    some manual interaction is okay from time to time.
>
> [YOCTO #5614]
>
> Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>

Nice.

> ---
>  meta/lib/oeqa/controllers/masterimage.py | 124 +++++++++++++++++++++++++++++++
>  meta/lib/oeqa/runtime/ssh.py             |   2 +
>  2 files changed, 126 insertions(+)
>  create mode 100644 meta/lib/oeqa/controllers/masterimage.py
>
> diff --git a/meta/lib/oeqa/controllers/masterimage.py b/meta/lib/oeqa/controllers/masterimage.py
> new file mode 100644
> index 0000000..028208b
> --- /dev/null
> +++ b/meta/lib/oeqa/controllers/masterimage.py
> @@ -0,0 +1,124 @@
> +import os
> +import bb
> +import traceback
> +import time
> +
> +import oeqa.targetcontrol
> +import oeqa.utils.sshcontrol as sshcontrol
> +import oeqa.utils.commands as commands
> +
> +class GenericEfi(oeqa.targetcontrol.SimpleRemoteTarget):
> +
> +    def __init__(self, d):
> +        # let our base class do the ip thing
> +        super(GenericEfi, self).__init__(d)
> +
> +        # test rootfs + kernel
> +        self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz')
> +        self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE"))
> +        if not os.path.isfile(self.rootfs):
> +            # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be
> +            # the same as the config with which the image was build, ie
> +            # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
> +            # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage
> +            bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \
> +                      \nExpected path: %s" % self.rootfs)

Couldn't testimage class add it?

> +        if not os.path.isfile(self.kernel):
> +            bb.fatal("No kernel found. Expected path: %s" % self.kernel)
> +
> +        # if the user knows what he's doing, then by all means...
> +        # test-rootfs.tar.gz and test-kernel are hardcoded names in other places
> +        # they really have to be used like that in commands though
> +        cmds = d.getVar("TEST_DEPLOY_CMDS", True)
> +        if cmds:
> +            self.deploy_cmds = cmds.split("\n")

Good.

> +        else:
> +            self.deploy_cmds = [
> +                'mount -L boot /boot',
> +                'mkdir -p /mnt/testrootfs',
> +                'mount -L testrootfs /mnt/testrootfs',
> +                'modprobe efivarfs',
> +                'mount -t efivarfs efivarfs /sys/firmware/efi/efivars',
> +                'cp ~/test-kernel /boot',
> +                'rm -rf /mnt/testrootfs/*',
> +                'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',

How target see those files? NFS?

> +                r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" > /sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'

Can you elaborate this? Seems too 'black magic' for me.

> +                ]
> +
> +        # master ssh connection
> +        self.master = None
> +
> +        # this is the name of the command that controls the power for a board
> +        # e.g: TEST_POWERCONTROL_CMD = "/home/user/myscripts/powercontrol.py ${MACHINE} what-ever-other-args-the-script-wants"
> +        # the command should take as the last argument "off" and "on" and "cycle" (off, on)
> +        self.powercontrol_cmd = d.getVar("TEST_POWERCONTROL_CMD", True) or None
> +        self.powercontrol_args = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or ""
> +        self.origenv = os.environ
> +        if self.powercontrol_cmd:
> +            if self.powercontrol_args:
> +                self.powercontrol_cmd = "%s %s" % (self.powercontrol_cmd, self.powercontrol_args)
> +            # the external script for controlling power might use ssh
> +            # ssh + keys means we need the original user env
> +            bborigenv = d.getVar("BB_ORIGENV", False) or {}
> +            for key in bborigenv:
> +                val = bborigenv.getVar(key, True)
> +                if val is not None:
> +                    self.origenv[key] = str(val)
> +            self.power_ctl("on")
> +
> +    def power_ctl(self, msg):
> +        if self.powercontrol_cmd:
> +            cmd = "%s %s" % (self.powercontrol_cmd, msg)
> +            commands.runCmd(cmd, preexec_fn=os.setsid, env=self.origenv)
> +
> +    def power_cycle(self, conn):
> +        if self.powercontrol_cmd:
> +            # be nice, don't just cut power
> +            conn.run("shutdown -h now")
> +            time.sleep(10)
> +            self.power_ctl("cycle")
> +        else:
> +            status, output = conn.run("reboot")
> +            if status != 0:
> +                bb.error("Failed rebooting target and no power control command defined. You need to manually reset the device.\n%s" % output)
> +
> +    def deploy(self):
> +        bb.plain("%s - deploying image on target" % self.pn)
> +        # base class just sets the ssh log file for us
> +        super(GenericEfi, self).deploy()
> +        self.master = sshcontrol.SSHControl(ip=self.ip, logfile=self.sshlog, timeout=600, port=self.port)
> +        try:
> +            self._deploy()
> +        except Exception as e:
> +            bb.fatal("Failed deploying test image: %s" % e)
> +
> +    def _deploy(self):
> +        # make sure we are in the right image
> +        status, output = self.master.run("cat /etc/masterimage")
> +        if status != 0:
> +            raise Exception("No ssh connectivity or target isn't running a master image.\n%s" % output)
> +
> +        # make sure these aren't mounted
> +        self.master.run("umount /boot; umount /mnt/testrootfs; umount /sys/firmware/efi/efivars;")
> +
> +        # from now on, every deploy cmd should return 0
> +        self.master.ignore_status = False
> +        self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz")
> +        self.master.copy_to(self.kernel, "~/test-kernel")
> +        for cmd in self.deploy_cmds:
> +            self.master.run(cmd)
> +
> +
> +    def start(self, params=None):
> +        bb.plain("%s - boot test image on target" % self.pn)
> +        self.power_cycle(self.master)
> +        # assuming the reboot worked, we need to wait a bit
> +        # there are better ways than a timeout but this should work for my purpose for now
> +        time.sleep(120)
> +        # we are live, set the ssh object for the target/test image
> +        self.connection = sshcontrol.SSHControl(self.ip, logfile=self.sshlog, port=self.port)
> +        bb.plain("%s - start running tests" % self.pn)
> +
> +    def stop(self):
> +        bb.plain("%s - reboot/powercycle target" % self.pn)
> +        self.power_cycle(self.connection)
> diff --git a/meta/lib/oeqa/runtime/ssh.py b/meta/lib/oeqa/runtime/ssh.py
> index 8c96020..e648660 100644
> --- a/meta/lib/oeqa/runtime/ssh.py
> +++ b/meta/lib/oeqa/runtime/ssh.py
> @@ -14,3 +14,5 @@ class SshTest(oeRuntimeTest):
>      def test_ssh(self):
>          (status, output) = self.target.run('uname -a')
>          self.assertEqual(status, 0, msg="SSH Test failed: %s" % output)
> +        (status, output) = self.target.run('cat /etc/masterimage')
> +        self.assertEqual(status, 1, msg="This isn't the right image  - /etc/masterimage shouldn't be here %s" % output)
> --
> 1.8.5.3
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Stanacar, StefanX - March 21, 2014, 8:57 a.m.
Hi Otavio,

On Thu, 2014-03-20 at 18:08 -0300, Otavio Salvador wrote:
> On Thu, Mar 20, 2014 at 1:29 PM, Stefan Stanacar
> <stefanx.stanacar@intel.com> wrote:

> > +        # test rootfs + kernel
> > +        self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz')
> > +        self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE"))
> > +        if not os.path.isfile(self.rootfs):
> > +            # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be
> > +            # the same as the config with which the image was build, ie
> > +            # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
> > +            # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage
> > +            bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \
> > +                      \nExpected path: %s" % self.rootfs)
> 
> Couldn't testimage class add it?
> 

Nope... testimage task can be triggered in two ways: from
testimage.bbclass or testimage-auto.bbclass, that means:
  - manually: you have already built your image, then add INHERIT +=
"testimage" in local.conf and run bitbake <image> -c testimage (the AB
does it this way). The testimage task does not have a depends on
do_rootfs, so it won't create a tar.gz rootfs if IMAGE_FSTYPES wasn't
added to local.conf at build time.
  - automatically: you add TEST_IMAGE = "1" in local.conf before bitbake
<image>. That's a hidden inherit in image.bbclass that inherits
testimage-auto that does depend on rootfs. 

We want to support both cases... So checking that tar.gz is
IMAGE_FSTYPES doesn't mean that the image actually was built with that.
Also, tar.gz might not be there although the image was built with that 
(The AB has two different build steps for build and running tests, and
in between the config can be overwritten).

> > +        if not os.path.isfile(self.kernel):
> > +            bb.fatal("No kernel found. Expected path: %s" % self.kernel)
> > +
> > +        # if the user knows what he's doing, then by all means...
> > +        # test-rootfs.tar.gz and test-kernel are hardcoded names in other places
> > +        # they really have to be used like that in commands though
> > +        cmds = d.getVar("TEST_DEPLOY_CMDS", True)
> > +        if cmds:
> > +            self.deploy_cmds = cmds.split("\n")
> 
> Good.
> 
> > +        else:
> > +            self.deploy_cmds = [
> > +                'mount -L boot /boot',
> > +                'mkdir -p /mnt/testrootfs',
> > +                'mount -L testrootfs /mnt/testrootfs',
> > +                'modprobe efivarfs',
> > +                'mount -t efivarfs efivarfs /sys/firmware/efi/efivars',
> > +                'cp ~/test-kernel /boot',
> > +                'rm -rf /mnt/testrootfs/*',
> > +                'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',
> 
> How target see those files? NFS?

Nope, see below in _deploy.
Those files are scp'ed from the build machine to the target.

      self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz")
      self.master.copy_to(self.kernel, "~/test-kernel")
      for cmd in self.deploy_cmds:
            self.master.run(cmd)

> 
> > +                r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" > /sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'
> 
> Can you elaborate this? Seems too 'black magic' for me.
> 

Okay so one of the reasons this GenericEfi target requires that the
master image deployed uses gummiboot is because of that.
gummiboot actually reads and obeys system EFI variables. [1]
One of those variables is LoaderEntryOneShot which tells it to boot a
label (kernel  + rootfs) once and only once so this how we know that we
always get back to the master/good image.


[1] "Some EFI variables control the loader or exported the loaders state
to the started operating system. The vendor UUID
4a67b082-0a4c-41cf-b6c7-440b29bb8c4f and the variable names are supposed
to be shared across all loaders implementations which follow this scheme
of configuration: "
http://freedesktop.org/wiki/Software/gummiboot/


Accessing the EFI vars subsystem is done in two ways:
  - old sysfs-efivars interface (CONFIG_EFI_VARS), populated
at /sys/firmware/efi/vars, which has some limitations (basically you
need another tool to read/write)
- new efivarfs interface (CONFIG_EFIVAR_FS), typically mounted like
this: mount -t efivarfs efivarfs /sys/firmware/efi/efivars
It was added in 3.8 intended as a replacement for the sysfs-efivars
interface, has no maximum per-variable size limitation and supports UEFI
Secure Boot variables.
You can read more about that at:
http://uefidk.intel.com/blog/accessing-uefi-variables-linux#sthash.2wI28Mkn.dpuf

The first four bytes are EFI_VARIABLE_NON_VOLATILE,
EFI_VARIABLE_BOOTSERVICE_ACCESS and EFI_VARIABLE_RUNTIME_ACCESS bits set
in the attribute bitmask, and the rest is just the output from:
[stefans@firebird ~]$ echo -en "test" | iconv -f ascii -t utf-16le |
hexdump -C
00000000  74 00 65 00 73 00 74 00                           |t.e.s.t.|
00000008
[stefans@firebird ~]$ 

The gummiboot loader config that gets installed with the master image
has two labels: boot and test and this is how we tell it that for the
next boot only we want the test label.

The simplest tool to read EFI vars I've found is efivar. I've used it
when debugging and playing with this stuff.
There is a recipe here:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=stefans/wip2&id=3df22b8ab36553052381458314b43b76cd1d3dff

I haven't submitted it because I don't know how many people need it.

HTH,

Cheers,
Stefan

> > +                ]
> > +
> > +        # master ssh connection
> > +        self.master = None
> > +
> > +        # this is the name of the command that controls the power for a board
> > +        # e.g: TEST_POWERCONTROL_CMD = "/home/user/myscripts/powercontrol.py ${MACHINE} what-ever-other-args-the-script-wants"
> > +        # the command should take as the last argument "off" and "on" and "cycle" (off, on)
> > +        self.powercontrol_cmd = d.getVar("TEST_POWERCONTROL_CMD", True) or None
> > +        self.powercontrol_args = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or ""
> > +        self.origenv = os.environ
> > +        if self.powercontrol_cmd:
> > +            if self.powercontrol_args:
> > +                self.powercontrol_cmd = "%s %s" % (self.powercontrol_cmd, self.powercontrol_args)
> > +            # the external script for controlling power might use ssh
> > +            # ssh + keys means we need the original user env
> > +            bborigenv = d.getVar("BB_ORIGENV", False) or {}
> > +            for key in bborigenv:
> > +                val = bborigenv.getVar(key, True)
> > +                if val is not None:
> > +                    self.origenv[key] = str(val)
> > +            self.power_ctl("on")
> > +
> > +    def power_ctl(self, msg):
> > +        if self.powercontrol_cmd:
> > +            cmd = "%s %s" % (self.powercontrol_cmd, msg)
> > +            commands.runCmd(cmd, preexec_fn=os.setsid, env=self.origenv)
> > +
> > +    def power_cycle(self, conn):
> > +        if self.powercontrol_cmd:
> > +            # be nice, don't just cut power
> > +            conn.run("shutdown -h now")
> > +            time.sleep(10)
> > +            self.power_ctl("cycle")
> > +        else:
> > +            status, output = conn.run("reboot")
> > +            if status != 0:
> > +                bb.error("Failed rebooting target and no power control command defined. You need to manually reset the device.\n%s" % output)
> > +
> > +    def deploy(self):
> > +        bb.plain("%s - deploying image on target" % self.pn)
> > +        # base class just sets the ssh log file for us
> > +        super(GenericEfi, self).deploy()
> > +        self.master = sshcontrol.SSHControl(ip=self.ip, logfile=self.sshlog, timeout=600, port=self.port)
> > +        try:
> > +            self._deploy()
> > +        except Exception as e:
> > +            bb.fatal("Failed deploying test image: %s" % e)
> > +
> > +    def _deploy(self):
> > +        # make sure we are in the right image
> > +        status, output = self.master.run("cat /etc/masterimage")
> > +        if status != 0:
> > +            raise Exception("No ssh connectivity or target isn't running a master image.\n%s" % output)
> > +
> > +        # make sure these aren't mounted
> > +        self.master.run("umount /boot; umount /mnt/testrootfs; umount /sys/firmware/efi/efivars;")
> > +
> > +        # from now on, every deploy cmd should return 0
> > +        self.master.ignore_status = False
> > +        self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz")
> > +        self.master.copy_to(self.kernel, "~/test-kernel")
> > +        for cmd in self.deploy_cmds:
> > +            self.master.run(cmd)
> > +
> > +
> > +    def start(self, params=None):
> > +        bb.plain("%s - boot test image on target" % self.pn)
> > +        self.power_cycle(self.master)
> > +        # assuming the reboot worked, we need to wait a bit
> > +        # there are better ways than a timeout but this should work for my purpose for now
> > +        time.sleep(120)
> > +        # we are live, set the ssh object for the target/test image
> > +        self.connection = sshcontrol.SSHControl(self.ip, logfile=self.sshlog, port=self.port)
> > +        bb.plain("%s - start running tests" % self.pn)
> > +
> > +    def stop(self):
> > +        bb.plain("%s - reboot/powercycle target" % self.pn)
> > +        self.power_cycle(self.connection)
> > diff --git a/meta/lib/oeqa/runtime/ssh.py b/meta/lib/oeqa/runtime/ssh.py
> > index 8c96020..e648660 100644
> > --- a/meta/lib/oeqa/runtime/ssh.py
> > +++ b/meta/lib/oeqa/runtime/ssh.py
> > @@ -14,3 +14,5 @@ class SshTest(oeRuntimeTest):
> >      def test_ssh(self):
> >          (status, output) = self.target.run('uname -a')
> >          self.assertEqual(status, 0, msg="SSH Test failed: %s" % output)
> > +        (status, output) = self.target.run('cat /etc/masterimage')
> > +        self.assertEqual(status, 1, msg="This isn't the right image  - /etc/masterimage shouldn't be here %s" % output)
> > --
> > 1.8.5.3
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 
>
Otavio Salvador - March 21, 2014, 8:07 p.m.
On Fri, Mar 21, 2014 at 5:57 AM, Stanacar, StefanX
<stefanx.stanacar@intel.com> wrote:
> On Thu, 2014-03-20 at 18:08 -0300, Otavio Salvador wrote:
>> On Thu, Mar 20, 2014 at 1:29 PM, Stefan Stanacar
>> <stefanx.stanacar@intel.com> wrote:
>
>> > +        # test rootfs + kernel
>> > +        self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz')
>> > +        self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE"))
>> > +        if not os.path.isfile(self.rootfs):
>> > +            # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be
>> > +            # the same as the config with which the image was build, ie
>> > +            # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
>> > +            # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage
>> > +            bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \
>> > +                      \nExpected path: %s" % self.rootfs)
>>
>> Couldn't testimage class add it?
>>
>
> Nope... testimage task can be triggered in two ways: from
> testimage.bbclass or testimage-auto.bbclass, that means:
>   - manually: you have already built your image, then add INHERIT +=
> "testimage" in local.conf and run bitbake <image> -c testimage (the AB
> does it this way). The testimage task does not have a depends on
> do_rootfs, so it won't create a tar.gz rootfs if IMAGE_FSTYPES wasn't
> added to local.conf at build time.
>   - automatically: you add TEST_IMAGE = "1" in local.conf before bitbake
> <image>. That's a hidden inherit in image.bbclass that inherits
> testimage-auto that does depend on rootfs.
>
> We want to support both cases... So checking that tar.gz is
> IMAGE_FSTYPES doesn't mean that the image actually was built with that.
> Also, tar.gz might not be there although the image was built with that
> (The AB has two different build steps for build and running tests, and
> in between the config can be overwritten).

Ok.

>> > +        if not os.path.isfile(self.kernel):
>> > +            bb.fatal("No kernel found. Expected path: %s" % self.kernel)
>> > +
>> > +        # if the user knows what he's doing, then by all means...
>> > +        # test-rootfs.tar.gz and test-kernel are hardcoded names in other places
>> > +        # they really have to be used like that in commands though
>> > +        cmds = d.getVar("TEST_DEPLOY_CMDS", True)
>> > +        if cmds:
>> > +            self.deploy_cmds = cmds.split("\n")
>>
>> Good.
>>
>> > +        else:
>> > +            self.deploy_cmds = [
>> > +                'mount -L boot /boot',
>> > +                'mkdir -p /mnt/testrootfs',
>> > +                'mount -L testrootfs /mnt/testrootfs',
>> > +                'modprobe efivarfs',
>> > +                'mount -t efivarfs efivarfs /sys/firmware/efi/efivars',
>> > +                'cp ~/test-kernel /boot',
>> > +                'rm -rf /mnt/testrootfs/*',
>> > +                'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',
>>
>> How target see those files? NFS?
>
> Nope, see below in _deploy.
> Those files are scp'ed from the build machine to the target.
>
>       self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz")
>       self.master.copy_to(self.kernel, "~/test-kernel")
>       for cmd in self.deploy_cmds:
>             self.master.run(cmd)

What happens if the image is bigger than RAM size?

>> > +                r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" > /sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'
>>
>> Can you elaborate this? Seems too 'black magic' for me.
>>
>
> Okay so one of the reasons this GenericEfi target requires that the
> master image deployed uses gummiboot is because of that.
> gummiboot actually reads and obeys system EFI variables. [1]
> One of those variables is LoaderEntryOneShot which tells it to boot a
> label (kernel  + rootfs) once and only once so this how we know that we
> always get back to the master/good image.

So it is misnamed. GummiBootEfi would be clearer.

> [1] "Some EFI variables control the loader or exported the loaders state
> to the started operating system. The vendor UUID
> 4a67b082-0a4c-41cf-b6c7-440b29bb8c4f and the variable names are supposed
> to be shared across all loaders implementations which follow this scheme
> of configuration: "
> http://freedesktop.org/wiki/Software/gummiboot/
>
>
> Accessing the EFI vars subsystem is done in two ways:
>   - old sysfs-efivars interface (CONFIG_EFI_VARS), populated
> at /sys/firmware/efi/vars, which has some limitations (basically you
> need another tool to read/write)
> - new efivarfs interface (CONFIG_EFIVAR_FS), typically mounted like
> this: mount -t efivarfs efivarfs /sys/firmware/efi/efivars
> It was added in 3.8 intended as a replacement for the sysfs-efivars
> interface, has no maximum per-variable size limitation and supports UEFI
> Secure Boot variables.
> You can read more about that at:
> http://uefidk.intel.com/blog/accessing-uefi-variables-linux#sthash.2wI28Mkn.dpuf
>
> The first four bytes are EFI_VARIABLE_NON_VOLATILE,
> EFI_VARIABLE_BOOTSERVICE_ACCESS and EFI_VARIABLE_RUNTIME_ACCESS bits set
> in the attribute bitmask, and the rest is just the output from:
> [stefans@firebird ~]$ echo -en "test" | iconv -f ascii -t utf-16le |
> hexdump -C
> 00000000  74 00 65 00 73 00 74 00                           |t.e.s.t.|
> 00000008
> [stefans@firebird ~]$

So please make those constants in the code and proper documented.

> The gummiboot loader config that gets installed with the master image
> has two labels: boot and test and this is how we tell it that for the
> next boot only we want the test label.
>
> The simplest tool to read EFI vars I've found is efivar. I've used it
> when debugging and playing with this stuff.
> There is a recipe here:
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=stefans/wip2&id=3df22b8ab36553052381458314b43b76cd1d3dff

I think it would be nice for debugging so in worse case send t to meta-oe

:-)

Patch

diff --git a/meta/lib/oeqa/controllers/masterimage.py b/meta/lib/oeqa/controllers/masterimage.py
new file mode 100644
index 0000000..028208b
--- /dev/null
+++ b/meta/lib/oeqa/controllers/masterimage.py
@@ -0,0 +1,124 @@ 
+import os
+import bb
+import traceback
+import time
+
+import oeqa.targetcontrol
+import oeqa.utils.sshcontrol as sshcontrol
+import oeqa.utils.commands as commands
+
+class GenericEfi(oeqa.targetcontrol.SimpleRemoteTarget):
+
+    def __init__(self, d):
+        # let our base class do the ip thing
+        super(GenericEfi, self).__init__(d)
+
+        # test rootfs + kernel
+        self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz')
+        self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE"))
+        if not os.path.isfile(self.rootfs):
+            # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be
+            # the same as the config with which the image was build, ie
+            # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz"
+            # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage
+            bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \
+                      \nExpected path: %s" % self.rootfs)
+        if not os.path.isfile(self.kernel):
+            bb.fatal("No kernel found. Expected path: %s" % self.kernel)
+
+        # if the user knows what he's doing, then by all means...
+        # test-rootfs.tar.gz and test-kernel are hardcoded names in other places
+        # they really have to be used like that in commands though
+        cmds = d.getVar("TEST_DEPLOY_CMDS", True)
+        if cmds:
+            self.deploy_cmds = cmds.split("\n")
+        else:
+            self.deploy_cmds = [
+                'mount -L boot /boot',
+                'mkdir -p /mnt/testrootfs',
+                'mount -L testrootfs /mnt/testrootfs',
+                'modprobe efivarfs',
+                'mount -t efivarfs efivarfs /sys/firmware/efi/efivars',
+                'cp ~/test-kernel /boot',
+                'rm -rf /mnt/testrootfs/*',
+                'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',
+                r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" > /sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'
+                ]
+
+        # master ssh connection
+        self.master = None
+
+        # this is the name of the command that controls the power for a board
+        # e.g: TEST_POWERCONTROL_CMD = "/home/user/myscripts/powercontrol.py ${MACHINE} what-ever-other-args-the-script-wants"
+        # the command should take as the last argument "off" and "on" and "cycle" (off, on)
+        self.powercontrol_cmd = d.getVar("TEST_POWERCONTROL_CMD", True) or None
+        self.powercontrol_args = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or ""
+        self.origenv = os.environ
+        if self.powercontrol_cmd:
+            if self.powercontrol_args:
+                self.powercontrol_cmd = "%s %s" % (self.powercontrol_cmd, self.powercontrol_args)
+            # the external script for controlling power might use ssh
+            # ssh + keys means we need the original user env
+            bborigenv = d.getVar("BB_ORIGENV", False) or {}
+            for key in bborigenv:
+                val = bborigenv.getVar(key, True)
+                if val is not None:
+                    self.origenv[key] = str(val)
+            self.power_ctl("on")
+
+    def power_ctl(self, msg):
+        if self.powercontrol_cmd:
+            cmd = "%s %s" % (self.powercontrol_cmd, msg)
+            commands.runCmd(cmd, preexec_fn=os.setsid, env=self.origenv)
+
+    def power_cycle(self, conn):
+        if self.powercontrol_cmd:
+            # be nice, don't just cut power
+            conn.run("shutdown -h now")
+            time.sleep(10)
+            self.power_ctl("cycle")
+        else:
+            status, output = conn.run("reboot")
+            if status != 0:
+                bb.error("Failed rebooting target and no power control command defined. You need to manually reset the device.\n%s" % output)
+
+    def deploy(self):
+        bb.plain("%s - deploying image on target" % self.pn)
+        # base class just sets the ssh log file for us
+        super(GenericEfi, self).deploy()
+        self.master = sshcontrol.SSHControl(ip=self.ip, logfile=self.sshlog, timeout=600, port=self.port)
+        try:
+            self._deploy()
+        except Exception as e:
+            bb.fatal("Failed deploying test image: %s" % e)
+
+    def _deploy(self):
+        # make sure we are in the right image
+        status, output = self.master.run("cat /etc/masterimage")
+        if status != 0:
+            raise Exception("No ssh connectivity or target isn't running a master image.\n%s" % output)
+
+        # make sure these aren't mounted
+        self.master.run("umount /boot; umount /mnt/testrootfs; umount /sys/firmware/efi/efivars;")
+
+        # from now on, every deploy cmd should return 0
+        self.master.ignore_status = False
+        self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz")
+        self.master.copy_to(self.kernel, "~/test-kernel")
+        for cmd in self.deploy_cmds:
+            self.master.run(cmd)
+
+
+    def start(self, params=None):
+        bb.plain("%s - boot test image on target" % self.pn)
+        self.power_cycle(self.master)
+        # assuming the reboot worked, we need to wait a bit
+        # there are better ways than a timeout but this should work for my purpose for now
+        time.sleep(120)
+        # we are live, set the ssh object for the target/test image
+        self.connection = sshcontrol.SSHControl(self.ip, logfile=self.sshlog, port=self.port)
+        bb.plain("%s - start running tests" % self.pn)
+
+    def stop(self):
+        bb.plain("%s - reboot/powercycle target" % self.pn)
+        self.power_cycle(self.connection)
diff --git a/meta/lib/oeqa/runtime/ssh.py b/meta/lib/oeqa/runtime/ssh.py
index 8c96020..e648660 100644
--- a/meta/lib/oeqa/runtime/ssh.py
+++ b/meta/lib/oeqa/runtime/ssh.py
@@ -14,3 +14,5 @@  class SshTest(oeRuntimeTest):
     def test_ssh(self):
         (status, output) = self.target.run('uname -a')
         self.assertEqual(status, 0, msg="SSH Test failed: %s" % output)
+        (status, output) = self.target.run('cat /etc/masterimage')
+        self.assertEqual(status, 1, msg="This isn't the right image  - /etc/masterimage shouldn't be here %s" % output)