[1/5] Makefile/set_versions: Allow poky.yaml to be autogenerated

Message ID 20220321174455.1189669-2-richard.purdie@linuxfoundation.org
State New
Headers show
Series Proposed docs version changes | expand

Commit Message

Richard Purdie March 21, 2022, 5:44 p.m. UTC
Use a script to generate the branch/tag information inside poky.yaml.

If the branch isn't a known release branch, include git magic to find
the closest matching release branch we know about.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 documentation/.gitignore                  |   1 +
 documentation/Makefile                    |   1 +
 documentation/{poky.yaml => poky.yaml.in} |   0
 documentation/set_versions.py             | 115 ++++++++++++++++++++++
 4 files changed, 117 insertions(+)
 rename documentation/{poky.yaml => poky.yaml.in} (100%)
 create mode 100755 documentation/set_versions.py

Comments

Quentin Schulz March 23, 2022, 9:59 a.m. UTC | #1
Hi Michael,

On 3/23/22 10:46, Michael Opdenacker via lists.yoctoproject.org wrote:
> Hi Richard,
> 
> On 3/21/22 18:44, Richard Purdie wrote:
>> Use a script to generate the branch/tag information inside poky.yaml.
>>
>> If the branch isn't a known release branch, include git magic to find
>> the closest matching release branch we know about.
>>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> ---
>>   documentation/.gitignore                  |   1 +
>>   documentation/Makefile                    |   1 +
>>   documentation/{poky.yaml => poky.yaml.in} |   0
>>   documentation/set_versions.py             | 115 ++++++++++++++++++++++
>>   4 files changed, 117 insertions(+)
>>   rename documentation/{poky.yaml => poky.yaml.in} (100%)
>>   create mode 100755 documentation/set_versions.py
> 
> A minor issue here: "make clean"  doesn't remove "documentation/poky.yaml" .
> 

This is technically not an issue because open("poky.yaml", "w") 
overwrites the file if it already exists.

Though, we should make sure the run-docs-build script in 
yocto-autobuilder-helper repo actually runs git clean -ffdx before 
checking out another branch otherwise there can be a conflict because 
poky.yaml exists locally and would be overridden/overwritten by 
poky.yaml from an earlier release and fail the checkout. IIRC Richard 
had that in his patch so we should be good.

> What do you prefer? I submit a fix later and we can merge the branch, or
> you propose an update, at least for this patch?

I feel like a fixup patch is just fine, does not seem to be a blocker?

Cheers,
Quentin

Patch

diff --git a/documentation/.gitignore b/documentation/.gitignore
index 35ead8af6..e5e2c1708 100644
--- a/documentation/.gitignore
+++ b/documentation/.gitignore
@@ -1,5 +1,6 @@ 
 _build/
 Pipfile.lock
+poky.yaml
 .vscode/
 */svg/*.png
 */svg/*.pdf
diff --git a/documentation/Makefile b/documentation/Makefile
index f04f381bd..bec53399c 100644
--- a/documentation/Makefile
+++ b/documentation/Makefile
@@ -57,4 +57,5 @@  all: html epub latexpdf
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %:
+	$(SOURCEDIR)/set_versions.py
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/documentation/poky.yaml b/documentation/poky.yaml.in
similarity index 100%
rename from documentation/poky.yaml
rename to documentation/poky.yaml.in
diff --git a/documentation/set_versions.py b/documentation/set_versions.py
new file mode 100755
index 000000000..266ccf464
--- /dev/null
+++ b/documentation/set_versions.py
@@ -0,0 +1,115 @@ 
+#!/usr/bin/env python3
+#
+# Add version information to poky.yaml based upon current git branch/tags
+#
+# Copyright Linux Foundation
+# Author: Richard Purdie <richard.purdie@linuxfoundation.org>
+#
+# SPDX-License-Identifier: MIT
+#
+
+
+import subprocess
+import collections
+import sys
+
+devbranch = "kirkstone"
+#devbranch = "langdale"
+ltsseries = ["kirkstone", "dunfell"]
+
+release_series = collections.OrderedDict()
+#release_series["langdale"] = "4.1"
+release_series["kirkstone"] = "4.0"
+release_series["honister"] = "3.4"
+release_series["hardknott"] = "3.3"
+release_series["gatesgarth"] = "3.2"
+release_series["dunfell"] = "3.1"
+
+ourversion = None
+ourseries = None
+ourbranch = None
+
+# Test tags exist and inform the user to fetch if not
+try:
+    subprocess.run(["git", "show", "yocto-3.4.2"], capture_output=True, check=True)
+except subprocess.CalledProcessError:
+    sys.exit("Please run 'git fetch --tags' before building the documentation")
+
+# Try and figure out what we are
+tags = subprocess.run(["git", "tag", "--points-at", "HEAD"], capture_output=True, text=True).stdout
+for t in tags.split():
+    if t.startswith("yocto-"):
+        ourversion = t[6:]
+
+if ourversion:
+    # We're a tagged release
+    components = ourversion.split(".")
+    baseversion = components[0] + "." + components[1]
+    for i in release_series:
+        if release_series[i] == baseversion:
+            ourseries = i
+            ourbranch = i
+else:
+    # We're floating on a branch
+    branch = subprocess.run(["git", "branch", "--show-current"], capture_output=True, text=True).stdout.strip()
+    ourbranch = branch
+    if branch != "master" and branch not in release_series:
+        possible_branches = []
+        for b in release_series.keys():
+            result = subprocess.run(["git", "show-ref", "heads/" + b], capture_output=True, text=True)
+            if result.returncode == 0:
+                possible_branches.append(b)
+                continue
+            result = subprocess.run(["git", "show-ref", "origin/" + b], capture_output=True, text=True)
+            if result.returncode == 0:
+                possible_branches.append("origin/" + b)
+        nearestbranch = subprocess.run('git show-branch master ' + ' '.join(possible_branches) + ' | grep "*" | grep -v "$(git rev-parse --abbrev-ref HEAD)" | head -n1', shell=True, capture_output=True, text=True).stdout
+        branch = nearestbranch.split('[')[1].split('~')[0]
+        print("Nearest release branch esimtated to be %s" % branch)
+    if branch == "master":
+        ourseries = devbranch
+    elif branch in release_series:
+        ourseries = branch
+    else:
+        sys.exit("Unknown series for branch %s" % branch)
+
+    previoustags = subprocess.run(["git", "tag", "--merged", "HEAD"], capture_output=True, text=True).stdout
+    previoustags = [t[6:] for t in previoustags.split() if t.startswith("yocto-" + release_series[ourseries])]
+    futuretags = subprocess.run(["git", "tag", "--merged", ourbranch], capture_output=True, text=True).stdout
+    futuretags = [t[6:] for t in futuretags.split() if t.startswith("yocto-" + release_series[ourseries])]
+
+    # Append .999 against the last known version
+    if len(previoustags) != len(futuretags):
+        ourversion = previoustags[-1] + ".999"
+    else:
+        ourversion = release_series[ourseries] + ".999"
+
+series = [k for k in release_series]
+previousseries = series[series.index(ourseries)+1:]
+lastlts = [k for k in previousseries if k in ltsseries]
+
+print("Version calculated to be %s" % ourversion)
+print("Release series calculated to be %s" % ourseries)
+
+replacements = {
+    "DISTRO" : ourversion,
+    "DISTRO_NAME_NO_CAP" : ourseries,
+    "DISTRO_NAME" : ourseries.capitalize(),
+    "DISTRO_NAME_NO_CAP_MINUS_ONE" : previousseries[0],
+    "DISTRO_NAME_NO_CAP_LTS" : lastlts[0],
+    "YOCTO_DOC_VERSION" : ourversion,
+    "DISTRO_REL_TAG" : "yocto-" + ourversion,
+}
+
+with open("poky.yaml.in", "r") as r, open("poky.yaml", "w") as w:
+    lines = r.readlines()
+    for line in lines:
+        data = line.split(":")
+        k = data[0].strip()
+        if k in replacements:
+            w.write("%s : \"%s\"\n" % (k, replacements[k]))
+        else:
+            w.write(line)
+
+print("poky.yaml generated from poky.yaml.in")
+