Patchwork [bitbake-devel,1/2] V2 Disk space monitoring

login
register
mail settings
Submitter Robert Yang
Date Jan. 10, 2012, 12:45 p.m.
Message ID <bb164b343a162cee39a0ba2f76a7ffb906bc5381.1326199289.git.liezhi.yang@windriver.com>
Download mbox | patch
Permalink /patch/18925/
State New
Headers show

Comments

Robert Yang - Jan. 10, 2012, 12:45 p.m.
Monitor disk availability and warn the user when the free disk space or
amount of free inode is running low, it is enabled when BB_DISKMON_DIRS
is set.

* Variable meanings(from meta-yocto/conf/local.conf.sample):

  # Set the directories to monitor for disk usage, if more than one
  # directories are in the same device, then only one directory would be
  # warned since the monitor is based on the device.
  #
  BB_DISKMON_DIRS = "${TMPDIR} ${DL_DIR} ${SSTATE_DIR}"
  #
  # Set the minimum amount of disk space to warn, the unit can be G,
  # M or K, but do not use GB, MB or KB (B is not needed).
  # The default vaule is 1G if not set.
  #
  #BB_DISKMON_MINSPACE = "1G"
  #
  # Set the minimum amount of inodes to warn, the unit can be G, M or K.
  # The default value is 1K if not set.
  #
  #BB_DISKMON_MININODES = "1K"
  #
  # Set the monitor frequency, the unit can be h(hour), m(minute) or
  # s(second), when no unit is specified, the default unit is s(second),
  # for example, "60" means 60 seconds.
  # The default value is 60s if not set.
  #
  #BB_DISKMON_INTERVAL = "60s"

* Problems:

  1) The disk space monitor is running in a sub process of
     bin/bitbake, so it can't use the same logger as bitbake(or I don't
     know how to use it, I've tried to use the logger.warn('some
     information'), but it would print nothing in the subprocess),
     another solution maybe use an independent logger in the subprocess,
     but it seems that it doesn't worth because of the two reasons:
     a) The logger of bin/bitbake can't control the monitor's
        independent logger
     b) The monitor doesn't need print many things

     It would be great if anyone could give me more instructions.

 2) The code is placed in the new file lib/bb/monitordisk.py, since it
    seems that it is more independent.

[YOCTO #1589]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/bin/bitbake           |   13 ++++
 bitbake/lib/bb/monitordisk.py |  153 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 166 insertions(+), 0 deletions(-)
 create mode 100644 bitbake/lib/bb/monitordisk.py
Richard Purdie - Jan. 10, 2012, 5:51 p.m.
On Tue, 2012-01-10 at 20:45 +0800, Robert Yang wrote:
> * Problems:
> 
>   1) The disk space monitor is running in a sub process of
>      bin/bitbake, so it can't use the same logger as bitbake(or I don't
>      know how to use it, I've tried to use the logger.warn('some
>      information'), but it would print nothing in the subprocess),
>      another solution maybe use an independent logger in the subprocess,
>      but it seems that it doesn't worth because of the two reasons:
>      a) The logger of bin/bitbake can't control the monitor's
>         independent logger
>      b) The monitor doesn't need print many things
> 
>      It would be great if anyone could give me more instructions.

This is an issue unfortunately since its fine to have the warnings about
low space but its not much use unless bitbake can act on the warnings
somehow (stop executing more tasks for example).

I agree with Chris' comments about being able to specify the warning
threshold per directory being monitored too.

So I think before this can merge its going to need some further work. I
can't give pointers to the right way to send log messages from the
subprocess right at the moment. Perhaps you can insert yourself as an
"idle" task into bitbake's "main loop" instead of running as a
subprocess? This would also make it simpler to stop executing more
tasks. There probably should be a way for the user to choose
"immediately abort, no new tasks, show warning" as the options for
running out of space.

Cheers,

Richard
Chris Larson - Jan. 10, 2012, 7:54 p.m.
On Tue, Jan 10, 2012 at 10:51 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Tue, 2012-01-10 at 20:45 +0800, Robert Yang wrote:
>> * Problems:
>>
>>   1) The disk space monitor is running in a sub process of
>>      bin/bitbake, so it can't use the same logger as bitbake(or I don't
>>      know how to use it, I've tried to use the logger.warn('some
>>      information'), but it would print nothing in the subprocess),
>>      another solution maybe use an independent logger in the subprocess,
>>      but it seems that it doesn't worth because of the two reasons:
>>      a) The logger of bin/bitbake can't control the monitor's
>>         independent logger

If the monitor spawns from the bitbake server process, then it can
send log messages to the standard bitbake UI process, and it wouldn't
need its own log handling.

Patch

diff --git a/bitbake/bin/bitbake b/bitbake/bin/bitbake
index c2e6822..6bc34a4 100755
--- a/bitbake/bin/bitbake
+++ b/bitbake/bin/bitbake
@@ -39,6 +39,8 @@  import bb.msg
 from bb import cooker
 from bb import ui
 from bb import server
+from bb import monitordisk
+from multiprocessing import Process
 
 __version__ = "1.15.0"
 logger = logging.getLogger("BitBake")
@@ -223,6 +225,13 @@  Default BBFILES are the .bb files in the current directory.""")
     server.saveConnectionDetails()
     server.detach(cooker_logfile)
 
+    # For disk space monitor
+    diskMonDirs = cooker.configuration.data.getVar("BB_DISKMON_DIRS", 1)
+    if diskMonDirs:
+        monitorProcess = Process(target=monitordisk.monitorDiskSpace, \
+            args=(diskMonDirs, cooker.configuration.data))
+        monitorProcess.start()
+
     # Should no longer need to ever reference cooker
     del cooker
 
@@ -237,9 +246,13 @@  Default BBFILES are the .bb files in the current directory.""")
         finally:
             bb.event.ui_queue = []
             server_connection.terminate()
+            # Stop the disk space monitor
+            if diskMonDirs:
+                monitorProcess.terminate()
     else:
         print("server address: %s, server port: %s" % (server.serverinfo.host, server.serverinfo.port))
 
+
     return 1
 
 if __name__ == "__main__":
diff --git a/bitbake/lib/bb/monitordisk.py b/bitbake/lib/bb/monitordisk.py
new file mode 100644
index 0000000..cb67bc2
--- /dev/null
+++ b/bitbake/lib/bb/monitordisk.py
@@ -0,0 +1,153 @@ 
+#!/usr/bin/env python
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Copyright (C) 2011 Robert Yang
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import time, os, logging, re, sys
+import bb
+
+def errExit(info):
+    print("ERROR: %s" % info)
+    print("       Disk space monitor will NOT be enabled")
+    sys.exit(1)
+
+def convertGMK(unit):
+
+    """ Convert the space unit G, M, K, the unit is case-insensitive """
+
+    unitG = re.match('([1-9][0-9]*)[gG]\s?$', unit)
+    if unitG:
+        return int(unitG.group(1)) * (1024 ** 3)
+    unitM = re.match('([1-9][0-9]*)[mM]\s?$', unit)
+    if unitM:
+        return int(unitM.group(1)) * (1024 ** 2)
+    unitK = re.match('([1-9][0-9]*)[kK]\s?$', unit)
+    if unitK:
+        return int(unitK.group(1)) * 1024
+    unitN = re.match('([1-9][0-9]*)\s?$', unit)
+    if unitN:
+        return int(unitN.group(1))
+    else:
+        return None
+
+def convertHMS(unit):
+
+    """ Convert the time unit h, m, s, the unit is case-sensitive """
+
+    unitH = re.match('([1-9][0-9]*)h\s?$', unit)
+    if unitH:
+        return int(unitH.group(1)) * 60 * 60
+    unitM = re.match('([1-9][0-9]*)m\s?$', unit)
+    if unitM:
+        return int(unitM.group(1)) * 60
+    unitS = re.match('([1-9][0-9]*)s?\s?$', unit)
+    if unitS:
+        return int(unitS.group(1))
+    else:
+        return None
+
+def getMountedDevice(path):
+
+    """ Get the device mounted at the path, uses /proc/mounts """
+
+    # Get the mount point of the filesystem containing path
+    # st_dev is the ID of device containing file
+    parentDevice = os.stat(path).st_dev
+    currentDevice = parentDevice
+    # When the current directory's device is different from the
+    # parrent's, then the current directory is a mount point
+    while parentDevice == currentDevice:
+        mountPoint = path
+        # Use dirname to get the parrent's directory
+        path = os.path.dirname(path)
+        # Reach the "/"
+        if path == mountPoint:
+            break
+        parentDevice= os.stat(path).st_dev
+
+    try:
+        with open("/proc/mounts", "r") as ifp:
+            for line in ifp:
+                procLines = line.rstrip('\n').split()
+                if procLines[1] == mountPoint:
+                    return procLines[0]
+    except EnvironmentError:
+        pass
+    return None
+
+def monitorDiskSpace(BBDirs, configuration):
+
+    """Monitor disk space availability, warning when the disk space or
+       the amount of inode is running low"""
+
+    BBSpace = configuration.getVar("BB_DISKMON_MINSPACE", 1)
+    if BBSpace:
+        minSpace = convertGMK(BBSpace)
+        if not minSpace:
+            errExit("Invalid BB_DISKMON_MINSPACE: %s" % BBSpace)
+    else:
+        # The default value is 1GB
+        minSpace = 1024 ** 3
+
+    BBInodes = configuration.getVar("BB_DISKMON_MININODES", 1)
+    if BBInodes:
+        minInodes = convertGMK(BBInodes)
+        if not minInodes:
+            errExit("Invalid BB_DISKMON_MININODES : %s" % BBInodes)
+    else:
+        # The default value is 1K
+        minInodes = 1024
+
+    BBInterval = configuration.getVar("BB_DISKMON_INTERVAL", 1)
+    if BBInterval:
+        # The unit is second, the BB_DISKMON_INTERVAL could be ns or n
+        interval = convertHMS(BBInterval)
+        if not interval:
+            errExit("Invalid BB_DISKMON_INTERVAL: %s" % BBInterval)
+    else:
+        # The default value is 60s
+        interval = 60
+
+    # Save the device IDs, need the ID to be unique (the dictionary's key is
+    # unique), so that when more than one directories are located in the same
+    # device, we just monitor it once
+    deviceDict = {}
+    for path in BBDirs.split():
+        path = os.path.realpath(path)
+        # mkdir for the directory since it may not exist, for example the
+        # DL_DIR may not exist at the very beginning
+        if not os.path.exists(path):
+            bb.utils.mkdirhier(path)
+        mountedDevice = getMountedDevice(path)
+        deviceDict[mountedDevice] = path
+    while True:
+        for mountedDevice in deviceDict:
+            st = os.statvfs(deviceDict[mountedDevice])
+            # The free space, float point number
+            freeSpace = st.f_bavail * st.f_frsize
+            if freeSpace < minSpace:
+                print("WARNING: The free space of %s is running low (%.3fGB left)" % (mountedDevice, freeSpace / 1024 / 1024 / 1024.0))
+
+            # The free inodes, float point number
+            freeInodes = st.f_favail
+            if freeInodes < minInodes:
+                print("WARNING: The free inode of %s is running low (%.3fK left)" % (mountedDevice, freeInodes / 1024.0))
+        try:
+            time.sleep(interval)
+        except (KeyboardInterrupt, SystemExit):
+            sys.exit(1)
+