[meta-oe,v3,2/2] protobuf-c: add patch for protobuf 3 compatibility

Submitted by Pascal Bach on June 11, 2019, 3:38 p.m. | Patch ID: 162114

Details

Message ID 20190611153848.30239-3-pascal.bach@siemens.com
State Superseded
Commit fb3bd9c06eef3659f3fbc0467c6fd5b5439278ee
Headers show

Commit Message

Pascal Bach June 11, 2019, 3:38 p.m.
There is no release yet of protobuf-c that is compatible with protobuf 3.
Master is already patched so this is just a straight backport of this patch.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
---
 .../protobuf-c/protobuf3-compatibility.patch       | 55 ++++++++++++++++++++++
 .../recipes-devtools/protobuf/protobuf-c_1.3.1.bb  |  4 +-
 2 files changed, 58 insertions(+), 1 deletion(-)
 create mode 100644 meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch

Patch hide | download patch | download mbox

diff --git a/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch b/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
new file mode 100644
index 000000000..35914952d
--- /dev/null
+++ b/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
@@ -0,0 +1,55 @@ 
+From 7456d1621223d425b8a3fd74e435a79c046169fb Mon Sep 17 00:00:00 2001
+From: Robert Edmonds <edmonds@users.noreply.github.com>
+Date: Wed, 10 Apr 2019 20:47:48 -0400
+Subject: [PATCH] t/generated-code2/cxx-generate-packed-data.cc: Fix build
+ failure on newer protobuf
+
+ Upstream-Status: Backport [https://github.com/protobuf-c/protobuf-c/pull/369]
+
+google::protobuf::Message::Reflection has been removed in newer versions
+of protobuf. The replacement is google::protobuf::Reflection.
+
+protobuf in commit 779f61c6a3ce02a119e28e802f229e61b69b9046 ("Integrate
+recent changes from google3.", from August 2008) changed the following
+in message.h:
+
+    @@ -336,7 +337,8 @@ class LIBPROTOBUF_EXPORT Message {
+
+       // Introspection ---------------------------------------------------
+
+    -  class Reflection;  // Defined below.
+    +  // Typedef for backwards-compatibility.
+    +  typedef google::protobuf::Reflection Reflection;
+
+The "typedef for backwards-compatibility" apparently lasted ten years
+until protobuf commit 6bbe197e9c1b6fc38cbdc45e3bf83fa7ced792a3
+("Down-integrate from google3.", from August 2018) which finally removed
+the typedef:
+
+    @@ -327,8 +344,6 @@ class LIBPROTOBUF_EXPORT Message : public MessageLite {
+
+       // Introspection ---------------------------------------------------
+
+    -  // Typedef for backwards-compatibility.
+    -  typedef google::protobuf::Reflection Reflection;
+
+This commit updates the only use of this typedef (in the test suite) to
+directly refer to the replacement, google::protobuf::Reflection. This
+fixes the build failure in the test suite.
+---
+ t/generated-code2/cxx-generate-packed-data.cc | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/t/generated-code2/cxx-generate-packed-data.cc b/t/generated-code2/cxx-generate-packed-data.cc
+index 4fd3e25..0865d2e 100644
+--- a/t/generated-code2/cxx-generate-packed-data.cc
++++ b/t/generated-code2/cxx-generate-packed-data.cc
+@@ -998,7 +998,7 @@ static void dump_test_packed_repeated_enum (void)
+ static void dump_test_unknown_fields (void)
+ {
+   EmptyMess mess;
+-  const google::protobuf::Message::Reflection *reflection = mess.GetReflection();
++  const google::protobuf::Reflection *reflection = mess.GetReflection();
+   google::protobuf::UnknownFieldSet *fs = reflection->MutableUnknownFields(&mess);
+
+ #if GOOGLE_PROTOBUF_VERSION >= 2001000
diff --git a/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb b/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
index 7ef030092..1d1782306 100644
--- a/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
+++ b/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
@@ -15,7 +15,9 @@  DEPENDS = "protobuf-native protobuf"
 PV .= "+git${SRCPV}"
 SRCREV = "269771b4b45d3aba04e59569f53600003db8d9ff"
 
-SRC_URI = "git://github.com/protobuf-c/protobuf-c.git"
+SRC_URI = "git://github.com/protobuf-c/protobuf-c.git \
+           file://protobuf3-compatibility.patch \
+          "
 
 S = "${WORKDIR}/git"
 

Comments

Martin Jansa June 12, 2019, 6:23 p.m.
It's not caused by this change, but if you're using protobuf-c aren't you
seeing random build failures due to race-condition between generated
test-full.pb.h and cxx-generate-packed-data.cc ?

I've rebuilt protobuf-c-native 100 times in 32 core system and got 4 failed
builds, each with slightly different errors (only first 5 shown for each):

BUILD-40/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
../git/t/generated-code2/cxx-generate-packed-data.cc:28:17: error: ‘foo’ is
not a namespace-name
../git/t/generated-code2/cxx-generate-packed-data.cc:28:20: error: expected
namespace-name before ‘;’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error:
‘TestEnumSmall’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error:
‘TestEnumSmall’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error:
‘TestEnumSmall’ does not name a type

BUILD-76/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
/usr/include/inttypes.h:266:1: error: expected initializer before ‘extern’
../git/t/generated-code2/cxx-generate-packed-data.cc:1169:1: error:
expected ‘}’ at end of input

BUILD-85/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
/usr/include/inttypes.h:266:1: error: expected ‘)’ before ‘extern’
../git/t/generated-code2/cxx-generate-packed-data.cc:42:1: error: a
function-definition is not allowed here before ‘{’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:64:1: error: a
function-definition is not allowed here before ‘{’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:70:1: error: a
function-definition is not allowed here before ‘{’ token

BUILD-86/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
./t/test-full.pb.h:21888:15: error: ‘s’ in namespace ‘std’ does not name a
type
../git/t/generated-code2/cxx-generate-packed-data.cc:1169:1: error:
expected ‘}’ at end of input

It was supposed to be fixed with;
https://github.com/protobuf-c/protobuf-c/commit/b0bb56303366e2c072ee38eb5f1f11251b07239c
but the fix either doesn't work at all or is actually causing this (I
haven't seen these failures with older protobuf-c-native).

Maybe we should switch from autotools to CMake when building protobuf-c.

There is also PR with meson support:
https://github.com/protobuf-c/protobuf-c/pull/340

If you're not really interested in protobuf-c-native then you can ignore
this, it's not caused by your change - I wasn't trying to hijack this
thread and block your change.

Cheers,

On Tue, Jun 11, 2019 at 5:39 PM Pascal Bach <pascal.bach@siemens.com> wrote:

> There is no release yet of protobuf-c that is compatible with protobuf 3.
> Master is already patched so this is just a straight backport of this
> patch.
>
> Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
> ---
>  .../protobuf-c/protobuf3-compatibility.patch       | 55
> ++++++++++++++++++++++
>  .../recipes-devtools/protobuf/protobuf-c_1.3.1.bb  |  4 +-
>  2 files changed, 58 insertions(+), 1 deletion(-)
>  create mode 100644
> meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
>
> diff --git
> a/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
> b/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
> new file mode 100644
> index 000000000..35914952d
> --- /dev/null
> +++
> b/meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch
> @@ -0,0 +1,55 @@
> +From 7456d1621223d425b8a3fd74e435a79c046169fb Mon Sep 17 00:00:00 2001
> +From: Robert Edmonds <edmonds@users.noreply.github.com>
> +Date: Wed, 10 Apr 2019 20:47:48 -0400
> +Subject: [PATCH] t/generated-code2/cxx-generate-packed-data.cc: Fix build
> + failure on newer protobuf
> +
> + Upstream-Status: Backport [
> https://github.com/protobuf-c/protobuf-c/pull/369]
> +
> +google::protobuf::Message::Reflection has been removed in newer versions
> +of protobuf. The replacement is google::protobuf::Reflection.
> +
> +protobuf in commit 779f61c6a3ce02a119e28e802f229e61b69b9046 ("Integrate
> +recent changes from google3.", from August 2008) changed the following
> +in message.h:
> +
> +    @@ -336,7 +337,8 @@ class LIBPROTOBUF_EXPORT Message {
> +
> +       // Introspection
> ---------------------------------------------------
> +
> +    -  class Reflection;  // Defined below.
> +    +  // Typedef for backwards-compatibility.
> +    +  typedef google::protobuf::Reflection Reflection;
> +
> +The "typedef for backwards-compatibility" apparently lasted ten years
> +until protobuf commit 6bbe197e9c1b6fc38cbdc45e3bf83fa7ced792a3
> +("Down-integrate from google3.", from August 2018) which finally removed
> +the typedef:
> +
> +    @@ -327,8 +344,6 @@ class LIBPROTOBUF_EXPORT Message : public
> MessageLite {
> +
> +       // Introspection
> ---------------------------------------------------
> +
> +    -  // Typedef for backwards-compatibility.
> +    -  typedef google::protobuf::Reflection Reflection;
> +
> +This commit updates the only use of this typedef (in the test suite) to
> +directly refer to the replacement, google::protobuf::Reflection. This
> +fixes the build failure in the test suite.
> +---
> + t/generated-code2/cxx-generate-packed-data.cc | 2 +-
> + 1 file changed, 1 insertion(+), 1 deletion(-)
> +
> +diff --git a/t/generated-code2/cxx-generate-packed-data.cc
> b/t/generated-code2/cxx-generate-packed-data.cc
> +index 4fd3e25..0865d2e 100644
> +--- a/t/generated-code2/cxx-generate-packed-data.cc
> ++++ b/t/generated-code2/cxx-generate-packed-data.cc
> +@@ -998,7 +998,7 @@ static void dump_test_packed_repeated_enum (void)
> + static void dump_test_unknown_fields (void)
> + {
> +   EmptyMess mess;
> +-  const google::protobuf::Message::Reflection *reflection =
> mess.GetReflection();
> ++  const google::protobuf::Reflection *reflection = mess.GetReflection();
> +   google::protobuf::UnknownFieldSet *fs =
> reflection->MutableUnknownFields(&mess);
> +
> + #if GOOGLE_PROTOBUF_VERSION >= 2001000
> diff --git a/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
> b/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
> index 7ef030092..1d1782306 100644
> --- a/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
> +++ b/meta-oe/recipes-devtools/protobuf/protobuf-c_1.3.1.bb
> @@ -15,7 +15,9 @@ DEPENDS = "protobuf-native protobuf"
>  PV .= "+git${SRCPV}"
>  SRCREV = "269771b4b45d3aba04e59569f53600003db8d9ff"
>
> -SRC_URI = "git://github.com/protobuf-c/protobuf-c.git"
> +SRC_URI = "git://github.com/protobuf-c/protobuf-c.git \
> +           file://protobuf3-compatibility.patch \
> +          "
>
>  S = "${WORKDIR}/git"
>
> --
> 2.11.0
>
> --
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel
>
Pascal Bach June 12, 2019, 8:08 p.m.
Hi Martin,

I haven’t observed this issue so far. But I haven’t rebuilt it 100 times.
I might find some time to have a look at the cmake or meson support and if it works submit a patch to switch the recipe to it.

Stay tuned.

Pascal

From: Martin Jansa <martin.jansa@gmail.com>

Sent: Mittwoch, 12. Juni 2019 20:23
To: Bach, Pascal (SI BP R&D ZG FW) <pascal.bach@siemens.com>
Cc: openembedded-devel <openembedded-devel@lists.openembedded.org>
Subject: Re: [oe] [meta-oe][PATCHv3 2/2] protobuf-c: add patch for protobuf 3 compatibility

It's not caused by this change, but if you're using protobuf-c aren't you seeing random build failures due to race-condition between generated test-full.pb.h and cxx-generate-packed-data.cc ?

I've rebuilt protobuf-c-native 100 times in 32 core system and got 4 failed builds, each with slightly different errors (only first 5 shown for each):

BUILD-40/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
../git/t/generated-code2/cxx-generate-packed-data.cc:28:17: error: ‘foo’ is not a namespace-name
../git/t/generated-code2/cxx-generate-packed-data.cc:28:20: error: expected namespace-name before ‘;’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error: ‘TestEnumSmall’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error: ‘TestEnumSmall’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:31:36: error: ‘TestEnumSmall’ does not name a type

BUILD-76/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
/usr/include/inttypes.h:266:1: error: expected initializer before ‘extern’
../git/t/generated-code2/cxx-generate-packed-data.cc:1169:1: error: expected ‘}’ at end of input
BUILD-85/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
/usr/include/inttypes.h:266:1: error: expected ‘)’ before ‘extern’
../git/t/generated-code2/cxx-generate-packed-data.cc:42:1: error: a function-definition is not allowed here before ‘{’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:64:1: error: a function-definition is not allowed here before ‘{’ token
../git/t/generated-code2/cxx-generate-packed-data.cc:70:1: error: a function-definition is not allowed here before ‘{’ token
BUILD-86/work/x86_64-linux/protobuf-c-native/1.3.1+gitAUTOINC+269771b4b4-r0/temp/log.do_compile
./t/test-full.pb.h:4:0: error: unterminated #ifndef
./t/test-full.pb.h:21888:15: error: ‘s’ in namespace ‘std’ does not name a type
../git/t/generated-code2/cxx-generate-packed-data.cc:1169:1: error: expected ‘}’ at end of input

It was supposed to be fixed with;
https://github.com/protobuf-c/protobuf-c/commit/b0bb56303366e2c072ee38eb5f1f11251b07239c<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c%2Fcommit%2Fb0bb56303366e2c072ee38eb5f1f11251b07239c&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157388338&sdata=aqCYxWpGVloQh7mki0O3p0Z0M%2BWXFuocYCi5rQsqxls%3D&reserved=0>
but the fix either doesn't work at all or is actually causing this (I haven't seen these failures with older protobuf-c-native).

Maybe we should switch from autotools to CMake when building protobuf-c.

There is also PR with meson support:
https://github.com/protobuf-c/protobuf-c/pull/340<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fprotobuf-c%2Fprotobuf-c%2Fpull%2F340&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157388338&sdata=8myitFPOaBfceG%2FdI2x9D0rHOtaswHoFkFyVixoQwKM%3D&reserved=0>

If you're not really interested in protobuf-c-native then you can ignore this, it's not caused by your change - I wasn't trying to hijack this thread and block your change.

Cheers,

On Tue, Jun 11, 2019 at 5:39 PM Pascal Bach <pascal.bach@siemens.com<mailto:pascal.bach@siemens.com>> wrote:
There is no release yet of protobuf-c that is compatible with protobuf 3.
Master is already patched so this is just a straight backport of this patch.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com<mailto:pascal.bach@siemens.com>>

---
 .../protobuf-c/protobuf3-compatibility.patch       | 55 ++++++++++++++++++++++
 .../recipes-devtools/protobuf/protobuf-c_1.3.1.bb<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fprotobuf-c_1.3.1.bb&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157398333&sdata=1Nlxk8IRzE7hhJxxXcuqsuI8dqFGVRiF50tunWvE6r4%3D&reserved=0>  |  4 +-
 2 files changed, 58 insertions(+), 1 deletion(-)
 create mode 100644 meta-oe/recipes-devtools/protobuf/protobuf-c/protobuf3-compatibility.patch

--
2.11.0

--
_______________________________________________
Openembedded-devel mailing list
Openembedded-devel@lists.openembedded.org<mailto:Openembedded-devel@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-devel<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openembedded.org%2Fmailman%2Flistinfo%2Fopenembedded-devel&data=02%7C01%7Cpascal.bach%40siemens.com%7C6ea32c7da6344dc9a8b508d6ef631478%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C636959606157428311&sdata=aolVOgT563U4Mo17Y%2FFH88K8DLSgNlZ%2BNtECnwqeFLc%3D&reserved=0>