[bitbake-devel,15/15] bitbake-user-manual: Add documentation for BB_LOGCONFIG

Submitted by Joshua Watt on March 9, 2020, 4:33 p.m. | Patch ID: 170907

Details

Message ID 20200309163353.15362-16-JPEWhacker@gmail.com
State New
Headers show

Commit Message

Joshua Watt March 9, 2020, 4:33 p.m.
Adds documentation describing how to use the BB_LOGCONFIG variable to
enable custom logging.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 .../bitbake-user-manual-execution.xml         | 97 +++++++++++++++++++
 .../bitbake-user-manual-ref-variables.xml     | 11 +++
 2 files changed, 108 insertions(+)

Patch hide | download patch | download mbox

diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
index 8f606676b4..3b31f748cc 100644
--- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
@@ -929,4 +929,101 @@ 
             section.
         </para>
     </section>
+
+    <section id="logging">
+        <title>Logging</title>
+        <para>
+            In addition to the standard command line option to control how
+            verbose builds are when execute, bitbake also supports user defined
+            configuration of the
+            <ulink url='https://docs.python.org/3/library/logging.html'>Python logging</ulink>
+            facilities through the
+            <link linkend="var-bb-BB_LOGCONFIG"><filename>BB_LOGCONFIG</filename></link>
+            variable. This variable defines a json or yaml
+            <ulink url='https://docs.python.org/3/library/logging.config.html'>logging configuration</ulink>
+            that will be intelligently merged into the default configuration.
+            The logging configuration is merged using the following rules:
+            <itemizedlist>
+                <listitem><para>
+                    The user defined configuration will completely replace the default
+                    configuration if top level key
+                    <filename>bitbake_merge</filename> is set to the value
+                    <filename>False</filename>. In this case, all other rules
+                    are ignored.
+                </para></listitem>
+                <listitem><para>
+                    The user configuration must have a top level
+                    <filename>version</filename> which must match the value of
+                    the default configuration.
+                </para></listitem>
+                <listitem><para>
+                    Any keys defined in the <filename>handlers</filename>,
+                    <filename>formatters</filename>, or <filename>filters</filename>,
+                    will be merged into the same section in the default
+                    configuration, with the user specified keys taking
+                    replacing a default one if there is a conflict. In
+                    practice, this means that if both the default configuration
+                    and user configuration specify a handler named
+                    <filename>myhandler</filename>, the user defined one will
+                    replace the default. To prevent the user from inadvertently
+                    replacing a default handler, formatter, or filter, all of
+                    the default ones are named with a prefix of
+                    "<filename>BitBake.</filename>"
+                </para></listitem>
+                <listitem><para>
+                    If a logger is defined by the user with the key
+                    <filename>bitbake_merge</filename> set to
+                    <filename>False</filename>, that logger will be completely
+                    replaced by user configuration. In this case, no other
+                    rules will apply to that logger.
+                </listitem></para>
+                <listitem><para>
+                    All user defined <filename>filter</filename> and
+                    <filename>handlers</filename> properties for a given logger
+                    will be merged with corresponding properties from the
+                    default logger. For example, if the user configuration adds
+                    a filter called <filename>myFilter</filename> to the
+                    <filename>BitBake.SigGen</filename>, and the default
+                    configuration adds a filter called
+                    <filename>BitBake.defaultFilter</filename>, both filters
+                    will be applied to the logger
+                </listitem></para>
+            </itemizedlist>
+        </para>
+
+        <para>
+            As an example, consider the following user logging configuration
+            file which logs all Hash Equivalence related messages of VERBOSE or
+            higher to a file called <filename>hashequiv.log</filename>
+            <literallayout class='monospaced'>
+    {
+        "version": 1,
+        "handlers": {
+            "autobuilderlog": {
+                "class": "logging.FileHandler",
+                "formatter": "logfileFormatter",
+                "level": "DEBUG",
+                "filename": "hashequiv.log",
+                "mode": "w"
+            }
+        },
+        "formatters": {
+                "logfileFormatter": {
+                    "format": "%(name)s: %(levelname)s: %(message)s"
+                }
+        },
+        "loggers": {
+            "BitBake.SigGen.HashEquiv": {
+                "level": "VERBOSE",
+                "handlers": ["autobuilderlog"]
+            },
+            "BitBake.RunQueue.HashEquiv": {
+                "level": "VERBOSE",
+                "handlers": ["autobuilderlog"]
+            }
+        }
+    }
+            </literallayout>
+        </para>
+    </section>
 </chapter>
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.xml
index bae01d90c0..c4bd1f2584 100644
--- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.xml
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.xml
@@ -539,6 +539,17 @@ 
             </glossdef>
         </glossentry>
 
+        <glossentry id='var-bb-BB_LOGCONFIG'><glossterm>BB_LOGCONFIG</glossterm>
+            <glossdef>
+                <para>
+                    Specifies the name of a config file that contains the user
+                    logging configuration. See
+                    <link linkend="logging">Logging</link> for additional
+                    information
+                </para>
+            </glossdef>
+        </glossentry>
+
         <glossentry id='var-bb-BB_LOGFMT'><glossterm>BB_LOGFMT</glossterm>
             <glossdef>
                 <para>

Comments

Rich Persaud March 12, 2020, 11:50 p.m.
On Mar 9, 2020, at 12:34, Joshua Watt <jpewhacker@gmail.com> wrote:
> 
> ´╗┐Adds documentation describing how to use the BB_LOGCONFIG variable to
> enable custom logging.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
> .../bitbake-user-manual-execution.xml         | 97 +++++++++++++++++++
> .../bitbake-user-manual-ref-variables.xml     | 11 +++
> 2 files changed, 108 insertions(+)
> 
> diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
> index 8f606676b4..3b31f748cc 100644
> --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
> +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
> @@ -929,4 +929,101 @@
>             section.
>         </para>
>     </section>
> +
> +    <section id="logging">
> +        <title>Logging</title>
> +        <para>
> +            In addition to the standard command line option to control how
> +            verbose builds are when execute, bitbake also supports user defined
> +            configuration of the
> +            <ulink url='https://docs.python.org/3/library/logging.html'>Python logging</ulink>
> +            facilities through the
> +            <link linkend="var-bb-BB_LOGCONFIG"><filename>BB_LOGCONFIG</filename></link>
> +            variable. This variable defines a json or yaml
> +            <ulink url='https://docs.python.org/3/library/logging.config.html'>logging configuration</ulink>
> +            that will be intelligently merged into the default configuration.
> +            The logging configuration is merged using the following rules:
> +            <itemizedlist>
> +                <listitem><para>
> +                    The user defined configuration will completely replace the default
> +                    configuration if top level key
> +                    <filename>bitbake_merge</filename> is set to the value
> +                    <filename>False</filename>. In this case, all other rules
> +                    are ignored.
> +                </para></listitem>
> +                <listitem><para>
> +                    The user configuration must have a top level
> +                    <filename>version</filename> which must match the value of
> +                    the default configuration.
> +                </para></listitem>
> +                <listitem><para>
> +                    Any keys defined in the <filename>handlers</filename>,
> +                    <filename>formatters</filename>, or <filename>filters</filename>,
> +                    will be merged into the same section in the default
> +                    configuration, with the user specified keys taking
> +                    replacing a default one if there is a conflict. In
> +                    practice, this means that if both the default configuration
> +                    and user configuration specify a handler named
> +                    <filename>myhandler</filename>, the user defined one will
> +                    replace the default. To prevent the user from inadvertently
> +                    replacing a default handler, formatter, or filter, all of
> +                    the default ones are named with a prefix of
> +                    "<filename>BitBake.</filename>"
> +                </para></listitem>
> +                <listitem><para>
> +                    If a logger is defined by the user with the key
> +                    <filename>bitbake_merge</filename> set to
> +                    <filename>False</filename>, that logger will be completely
> +                    replaced by user configuration. In this case, no other
> +                    rules will apply to that logger.
> +                </listitem></para>
> +                <listitem><para>
> +                    All user defined <filename>filter</filename> and
> +                    <filename>handlers</filename> properties for a given logger
> +                    will be merged with corresponding properties from the
> +                    default logger. For example, if the user configuration adds
> +                    a filter called <filename>myFilter</filename> to the
> +                    <filename>BitBake.SigGen</filename>, and the default
> +                    configuration adds a filter called
> +                    <filename>BitBake.defaultFilter</filename>, both filters
> +                    will be applied to the logger
> +                </listitem></para>
> +            </itemizedlist>
> +        </para>
> +
> +        <para>
> +            As an example, consider the following user logging configuration
> +            file which logs all Hash Equivalence related messages of VERBOSE or
> +            higher to a file called <filename>hashequiv.log</filename>
> +            <literallayout class='monospaced'>
> +    {
> +        "version": 1,
> +        "handlers": {
> +            "autobuilderlog": {
> +                "class": "logging.FileHandler",
> +                "formatter": "logfileFormatter",
> +                "level": "DEBUG",
> +                "filename": "hashequiv.log",
> +                "mode": "w"
> +            }
> +        },
> +        "formatters": {
> +                "logfileFormatter": {
> +                    "format": "%(name)s: %(levelname)s: %(message)s"

What's the syntax for including a timestamp in the log message?  Or is that implicit?

Rich
Joshua Watt March 13, 2020, 1:42 a.m.
On Thu, Mar 12, 2020 at 6:50 PM Rich Persaud <rp@stacktrust.org> wrote:
>
> On Mar 9, 2020, at 12:34, Joshua Watt <jpewhacker@gmail.com> wrote:
> >
> > ´╗┐Adds documentation describing how to use the BB_LOGCONFIG variable to
> > enable custom logging.
> >
> > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > ---
> > .../bitbake-user-manual-execution.xml         | 97 +++++++++++++++++++
> > .../bitbake-user-manual-ref-variables.xml     | 11 +++
> > 2 files changed, 108 insertions(+)
> >
> > diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
> > index 8f606676b4..3b31f748cc 100644
> > --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
> > +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.xml
> > @@ -929,4 +929,101 @@
> >             section.
> >         </para>
> >     </section>
> > +
> > +    <section id="logging">
> > +        <title>Logging</title>
> > +        <para>
> > +            In addition to the standard command line option to control how
> > +            verbose builds are when execute, bitbake also supports user defined
> > +            configuration of the
> > +            <ulink url='https://docs.python.org/3/library/logging.html'>Python logging</ulink>
> > +            facilities through the
> > +            <link linkend="var-bb-BB_LOGCONFIG"><filename>BB_LOGCONFIG</filename></link>
> > +            variable. This variable defines a json or yaml
> > +            <ulink url='https://docs.python.org/3/library/logging.config.html'>logging configuration</ulink>
> > +            that will be intelligently merged into the default configuration.
> > +            The logging configuration is merged using the following rules:
> > +            <itemizedlist>
> > +                <listitem><para>
> > +                    The user defined configuration will completely replace the default
> > +                    configuration if top level key
> > +                    <filename>bitbake_merge</filename> is set to the value
> > +                    <filename>False</filename>. In this case, all other rules
> > +                    are ignored.
> > +                </para></listitem>
> > +                <listitem><para>
> > +                    The user configuration must have a top level
> > +                    <filename>version</filename> which must match the value of
> > +                    the default configuration.
> > +                </para></listitem>
> > +                <listitem><para>
> > +                    Any keys defined in the <filename>handlers</filename>,
> > +                    <filename>formatters</filename>, or <filename>filters</filename>,
> > +                    will be merged into the same section in the default
> > +                    configuration, with the user specified keys taking
> > +                    replacing a default one if there is a conflict. In
> > +                    practice, this means that if both the default configuration
> > +                    and user configuration specify a handler named
> > +                    <filename>myhandler</filename>, the user defined one will
> > +                    replace the default. To prevent the user from inadvertently
> > +                    replacing a default handler, formatter, or filter, all of
> > +                    the default ones are named with a prefix of
> > +                    "<filename>BitBake.</filename>"
> > +                </para></listitem>
> > +                <listitem><para>
> > +                    If a logger is defined by the user with the key
> > +                    <filename>bitbake_merge</filename> set to
> > +                    <filename>False</filename>, that logger will be completely
> > +                    replaced by user configuration. In this case, no other
> > +                    rules will apply to that logger.
> > +                </listitem></para>
> > +                <listitem><para>
> > +                    All user defined <filename>filter</filename> and
> > +                    <filename>handlers</filename> properties for a given logger
> > +                    will be merged with corresponding properties from the
> > +                    default logger. For example, if the user configuration adds
> > +                    a filter called <filename>myFilter</filename> to the
> > +                    <filename>BitBake.SigGen</filename>, and the default
> > +                    configuration adds a filter called
> > +                    <filename>BitBake.defaultFilter</filename>, both filters
> > +                    will be applied to the logger
> > +                </listitem></para>
> > +            </itemizedlist>
> > +        </para>
> > +
> > +        <para>
> > +            As an example, consider the following user logging configuration
> > +            file which logs all Hash Equivalence related messages of VERBOSE or
> > +            higher to a file called <filename>hashequiv.log</filename>
> > +            <literallayout class='monospaced'>
> > +    {
> > +        "version": 1,
> > +        "handlers": {
> > +            "autobuilderlog": {
> > +                "class": "logging.FileHandler",
> > +                "formatter": "logfileFormatter",
> > +                "level": "DEBUG",
> > +                "filename": "hashequiv.log",
> > +                "mode": "w"
> > +            }
> > +        },
> > +        "formatters": {
> > +                "logfileFormatter": {
> > +                    "format": "%(name)s: %(levelname)s: %(message)s"
>
> What's the syntax for including a timestamp in the log message?  Or is that implicit?

It's not implicit, but you can add it easily with %(asctime)s. There
are also several other options:
https://docs.python.org/3/library/logging.html#logrecord-attributes

>
> Rich