[8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

Submitted by yann.dirson@blade-group.com on Sept. 21, 2020, 2:15 p.m. | Patch ID: 176678

Details

Message ID 20200921141544.3145189-9-yann@blade-group.com
State Master Next
Commit 9c9a7f53e11b67b406506aaca453179871639f56
Headers show

Commit Message

yann.dirson@blade-group.com Sept. 21, 2020, 2:15 p.m.
From: Yann Dirson <yann@blade-group.com>

---
 meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
 .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
 2 files changed, 145 insertions(+)
 create mode 100644 meta/classes/systemtap.bbclass
 create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb

Patch hide | download patch | download mbox

diff --git a/meta/classes/systemtap.bbclass b/meta/classes/systemtap.bbclass
new file mode 100644
index 0000000000..928202c1ff
--- /dev/null
+++ b/meta/classes/systemtap.bbclass
@@ -0,0 +1,74 @@ 
+# A bbclass to cross-build systemtap (.stp) scripts and install them
+# suitably for on-target usage with staprun.  All compiled .ko files
+# are installed directly in /lib/modules/${KERNEL_VERSION}/systemtap/
+# so their names must be unique.
+
+# FIXME:
+# - cannot handle building a probe several times with different args
+
+# verbosity flags easy to override and amend
+STAP_FLAGS = "-v"
+#STAP_FLAGS += "-B V=1"
+
+# set to "1" to avoid failing on first error
+STAP_WARNONLY = "0"
+
+# Should be absolute (use ${S}, ${WORKDIR}, etc)
+# .stp scripts to be compiled and installed.
+STAP_SCRIPTS ??= ""
+# subset of STAP_SCRIPTS not to compile (exclusion from glob pattern)
+STAP_EXCLUDE_SCRIPTS ??= ""
+
+STAP_OUT ??= "${B}/systemtap"
+
+inherit module-base
+
+DEPENDS += "systemtap-native"
+
+STAPCMD = "stap -p4 ${STAP_FLAGS}"
+STAPCMD += "-a '${ARCH}' -B CROSS_COMPILE='${CROSS_COMPILE}'"
+STAPCMD += "-r '${STAGING_KERNEL_BUILDDIR}'"
+
+# FIXME: most of those should be in the scope of systemtap-native
+STAPCMD += "--sysroot='${RECIPE_SYSROOT}'"
+STAPCMD += "-R '${RECIPE_SYSROOT_NATIVE}${datadir}/systemtap/runtime'"
+STAPCMD += "-I '${RECIPE_SYSROOT_NATIVE}${datadir}/systemtap/tapset'"
+
+## FIXME workaround LDFLAGS not being made for ld
+TARGET_LDFLAGS="-O1 --hash-style=gnu --as-needed"
+
+do_compile_append() {
+    rm -rf ${STAP_OUT}
+    mkdir -p ${STAP_OUT}
+    # expand glob patterns
+    stap_scripts=$(echo ${STAP_SCRIPTS})
+    stap_exclude_scripts=$(echo ${STAP_EXCLUDE_SCRIPTS})
+    for s in ${stap_scripts}; do
+        m=$(basename ${s%.stp})
+        m=$(printf $m | tr -c '[:alnum:]' _)
+        case " ${stap_exclude_scripts} " in
+          *\ $s\ *) continue ;;
+        esac
+        eval extraflags=\${STAP_EXTRAFLAGS_$m}
+        (
+            cd ${STAP_OUT}
+            [ ! -e "$m.ko" ] || bbwarn "compiling $s: $m.ko already exists"
+            if ! ${STAPCMD} ${extraflags} -m "$m" "$s"; then
+                if [ "${STAP_WARNONLY}" = 1 ]; then
+                    bbwarn "build failure for $s"
+                else
+                    bbfatal "build failure for $s"
+                fi
+            fi
+        )
+    done
+}
+
+do_install_append() {
+    DST=${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/systemtap
+    install -d -m755 ${DST}
+    install -m644 ${STAP_OUT}/*.ko ${DST}/
+}
+
+FILES_${PN} += "${nonarch_base_libdir}/modules"
+RRECOMMENDS_${PN} += "systemtap-runtime"
diff --git a/meta/recipes-kernel/systemtap/systemtap-demo_git.bb b/meta/recipes-kernel/systemtap/systemtap-demo_git.bb
new file mode 100644
index 0000000000..f6b6c40cc1
--- /dev/null
+++ b/meta/recipes-kernel/systemtap/systemtap-demo_git.bb
@@ -0,0 +1,71 @@ 
+require systemtap_git.inc
+
+# FIXME just to avoid looking for patches we don't care about
+SRC_URI = "git://sourceware.org/git/systemtap.git"
+
+inherit systemtap
+
+STAP_SCRIPTS = "\
+ ${S}/testsuite/systemtap.examples/*/*.stp \
+"
+
+# Many scripts have specific needs, so we don't really care if they don't build,
+# and there are too many of them to exclude them manually, so we just tell the build
+# to issue warnings on build failures:
+# - specific kernel symbols, depending on specific kernel config options
+# - files on disk for userspace instrumentation
+# - FIXME but also plain oversights of this packaging (kernel.trace ?)
+STAP_WARNONLY = "1"
+
+export STAP_EXTRAFLAGS_badname = "-g"
+export STAP_EXTRAFLAGS_keyhack = "-g"
+export STAP_EXTRAFLAGS_eatmydata = "-g"
+export STAP_EXTRAFLAGS_ttyspy = "-g"
+
+# those require parameters
+STAP_EXCLUDE_SCRIPTS = "\
+ ${S}/testsuite/systemtap.examples/general/callgraph.stp \
+ ${S}/testsuite/systemtap.examples/general/eventcount.stp \
+ ${S}/testsuite/systemtap.examples/general/func_time_stats.stp \
+ ${S}/testsuite/systemtap.examples/general/para-callgraph-verbose.stp \
+ ${S}/testsuite/systemtap.examples/general/para-callgraph.stp \
+ ${S}/testsuite/systemtap.examples/general/py2example.stp \
+ ${S}/testsuite/systemtap.examples/general/py3example.stp \
+ ${S}/testsuite/systemtap.examples/general/regex.stp \
+ ${S}/testsuite/systemtap.examples/general/sizeof.stp \
+ ${S}/testsuite/systemtap.examples/general/sizeof_interactive.stp \
+ ${S}/testsuite/systemtap.examples/general/varwatch.stp \
+ ${S}/testsuite/systemtap.examples/general/watchdog.stp \
+ ${S}/testsuite/systemtap.examples/general/whythefail.stp \
+ ${S}/testsuite/systemtap.examples/io/deviceseeks.stp \
+ ${S}/testsuite/systemtap.examples/io/inodewatch.stp \
+ ${S}/testsuite/systemtap.examples/io/inodewatch2.stp \
+ ${S}/testsuite/systemtap.examples/io/iostat-scsi.stp \
+ ${S}/testsuite/systemtap.examples/io/traceio2.stp \
+ ${S}/testsuite/systemtap.examples/locks/bkl.stp \
+ ${S}/testsuite/systemtap.examples/memory/hw_watch_addr.stp \
+ ${S}/testsuite/systemtap.examples/memory/hw_watch_sym.stp \
+ ${S}/testsuite/systemtap.examples/network/connect_stat.stp \
+ ${S}/testsuite/systemtap.examples/network/netfilter_drop.stp \
+ ${S}/testsuite/systemtap.examples/process/chng_cpu.stp \
+ ${S}/testsuite/systemtap.examples/process/errsnoop.stp \
+ ${S}/testsuite/systemtap.examples/process/migrate.stp \
+ ${S}/testsuite/systemtap.examples/process/pfiles.stp \
+ ${S}/testsuite/systemtap.examples/process/plimit.stp \
+ ${S}/testsuite/systemtap.examples/process/psig.stp \
+ ${S}/testsuite/systemtap.examples/process/pstrace_exec.stp \
+ ${S}/testsuite/systemtap.examples/process/sigmon.stp \
+ ${S}/testsuite/systemtap.examples/profiling/fntimes.stp \
+ ${S}/testsuite/systemtap.examples/profiling/functioncallcount.stp \
+ ${S}/testsuite/systemtap.examples/profiling/graphcall.stp \
+ ${S}/testsuite/systemtap.examples/profiling/linetimes.stp \
+ ${S}/testsuite/systemtap.examples/profiling/periodic.stp \
+ ${S}/testsuite/systemtap.examples/profiling/sched_switch.stp \
+"
+
+# FIXME what's the problem with those ?
+STAP_EXCLUDE_SCRIPTS += "\
+ ${S}/testsuite/systemtap.examples/interrupt/interrupts-by-dev.stp \
+"
+
+B = "${WORKDIR}/build"

Comments

Richard Purdie Sept. 21, 2020, 7:46 p.m.
On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> From: Yann Dirson <yann@blade-group.com>
> 
> ---
>  meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
>  .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
>  2 files changed, 145 insertions(+)
>  create mode 100644 meta/classes/systemtap.bbclass
>  create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb

I'm afraid this patch is going to throw warnings all over the
autobuilder, e.g.:

https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488

(and there will be more).

We are trying to ensure all warnings are solved so that any that do
show are real issues and this patch would significantly distract from
that.

I'm also wary that we're now in feature freeze for release so perhaps
these changes should wait until 3.3?

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142694): https://lists.openembedded.org/g/openembedded-core/message/142694
Mute This Topic: https://lists.openembedded.org/mt/76990992/3617530
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-
yann.dirson@blade-group.com Sept. 21, 2020, 9:23 p.m.
Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
richard.purdie@linuxfoundation.org> a écrit :

> On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > From: Yann Dirson <yann@blade-group.com>
> >
> > ---
> >  meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
> >  .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
> >  2 files changed, 145 insertions(+)
> >  create mode 100644 meta/classes/systemtap.bbclass
> >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb
>
> I'm afraid this patch is going to throw warnings all over the
> autobuilder, e.g.:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
>
> (and there will be more).
>
> We are trying to ensure all warnings are solved so that any that do
> show are real issues and this patch would significantly distract from
> that.
>

OK, we could default to built just the handful of probes that should always
build,
and give instructions for those willing to use the current behaviour.

I'm also wary that we're now in feature freeze for release so perhaps
> these changes should wait until 3.3?
>

OK, I can reroll a series with just the fixes.

For the rest should I just wait for the gatesgarth branch to be created and
resubmit then ?
Richard Purdie Sept. 21, 2020, 10:50 p.m.
On Mon, 2020-09-21 at 23:23 +0200, Yann Dirson wrote:
> 
> 
> Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
> richard.purdie@linuxfoundation.org> a écrit :
> > On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > > From: Yann Dirson <yann@blade-group.com>
> > > 
> > > ---
> > >  meta/classes/systemtap.bbclass                | 74
> > +++++++++++++++++++
> > >  .../systemtap/systemtap-demo_git.bb           | 71
> > ++++++++++++++++++
> > >  2 files changed, 145 insertions(+)
> > >  create mode 100644 meta/classes/systemtap.bbclass
> > >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-
> > demo_git.bb
> > 
> > I'm afraid this patch is going to throw warnings all over the
> > autobuilder, e.g.:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
> > 
> > (and there will be more).
> > 
> > We are trying to ensure all warnings are solved so that any that do
> > show are real issues and this patch would significantly distract
> > from
> > that.
> 
> OK, we could default to built just the handful of probes that should
> always build, and give instructions for those willing to use the
> current behaviour.

I think that is going to be safer and we could grow into more examples
if/as/when we van build them.

> 
> > I'm also wary that we're now in feature freeze for release so
> > perhaps
> > these changes should wait until 3.3?
> 
> OK, I can reroll a series with just the fixes.

That would be great.

> For the rest should I just wait for the gatesgarth branch to be
> created and resubmit then ?

Once we start taking new work on master, yes.

Also, for a new recipe like this we need a maintainers entry, else we
see:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1377

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#142701): https://lists.openembedded.org/g/openembedded-core/message/142701
Mute This Topic: https://lists.openembedded.org/mt/76990992/3617530
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-
Bruce Ashfield Sept. 22, 2020, 12:41 a.m.
On Mon, Sep 21, 2020 at 6:51 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2020-09-21 at 23:23 +0200, Yann Dirson wrote:
> >
> >
> > Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
> > richard.purdie@linuxfoundation.org> a écrit :
> > > On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > > > From: Yann Dirson <yann@blade-group.com>
> > > >
> > > > ---
> > > >  meta/classes/systemtap.bbclass                | 74
> > > +++++++++++++++++++
> > > >  .../systemtap/systemtap-demo_git.bb           | 71
> > > ++++++++++++++++++
> > > >  2 files changed, 145 insertions(+)
> > > >  create mode 100644 meta/classes/systemtap.bbclass
> > > >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-
> > > demo_git.bb
> > >
> > > I'm afraid this patch is going to throw warnings all over the
> > > autobuilder, e.g.:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
> > >
> > > (and there will be more).
> > >
> > > We are trying to ensure all warnings are solved so that any that do
> > > show are real issues and this patch would significantly distract
> > > from
> > > that.
> >
> > OK, we could default to built just the handful of probes that should
> > always build, and give instructions for those willing to use the
> > current behaviour.
>
> I think that is going to be safer and we could grow into more examples
> if/as/when we van build them.

Also, as the person that usually picks up systemtap breakage first
on our new kernel versions .. we need to make sure that the reworked
recipe and class pass the oe-selftest for stap.

It is doing on target compilation, so ideally, that test should be expanded
to test bot the cross and on-target variants.

Cheers,

Bruce

>
> >
> > > I'm also wary that we're now in feature freeze for release so
> > > perhaps
> > > these changes should wait until 3.3?
> >
> > OK, I can reroll a series with just the fixes.
>
> That would be great.
>
> > For the rest should I just wait for the gatesgarth branch to be
> > created and resubmit then ?
>
> Once we start taking new work on master, yes.
>
> Also, for a new recipe like this we need a maintainers entry, else we
> see:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1377
>
> Cheers,
>
> Richard
>
>
>
> 
>