mbox

[0/3] Use frozen flag for cargo build

Message ID cover.1690795930.git.frederic.martinsons@gmail.com
State Not Applicable, archived
Headers show

Pull-request

https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock

Message

Frédéric Martinsons July 31, 2023, 9:44 a.m. UTC
From: Frederic Martinsons <frederic.martinsons@gmail.com>

This patch series force the usage of --frozen instead of --offline during cargo build.
This has the advantage to be sure that Cargo.lock file will not be modified.
Moreover, raise a clear error when Cargo.lock is not present.
For this to work, we must modify ourself the Cargo.lock before building
in case there are git repo that have been patched.

The last commit is to handle the specific case of rust-hello-world recipe.
If this recipe is no more necessary, we can drop this last patch.

The following changes since commit 9b5b850d6a6982bb8ff14dcfbb6769b293638293:

  libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)

are available in the Git repository at:

  https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock

Frederic Martinsons (3):
  cargo.bbclass: Use --frozen flag for cargo operations
  cargo_common.bbclass: Handle Cargo.lock modifications for git
    dependencies
  cargo.bbclass: Offer a way to use --offline instead of --frozen

 meta/classes-recipe/cargo.bbclass             |  9 +++-
 meta/classes-recipe/cargo_common.bbclass      | 43 +++++++++++++++++++
 .../rust-example/rust-hello-world_git.bb      |  2 +
 3 files changed, 53 insertions(+), 1 deletion(-)

Comments

Alexander Kanavin July 31, 2023, 5:48 p.m. UTC | #1
I think you need to better describe the benefits. What are the
improvements? Can we just stay with --offline?

There's lots of intricate code added around cargo.lock handling, with
special casing to revert to previous behavior via new variable, but
I'm struggling to understand: to what end?

Alex





On Mon, 31 Jul 2023 at 11:44, Frederic Martinsons
<frederic.martinsons@gmail.com> wrote:
>
> From: Frederic Martinsons <frederic.martinsons@gmail.com>
>
> This patch series force the usage of --frozen instead of --offline during cargo build.
> This has the advantage to be sure that Cargo.lock file will not be modified.
> Moreover, raise a clear error when Cargo.lock is not present.
> For this to work, we must modify ourself the Cargo.lock before building
> in case there are git repo that have been patched.
>
> The last commit is to handle the specific case of rust-hello-world recipe.
> If this recipe is no more necessary, we can drop this last patch.
>
> The following changes since commit 9b5b850d6a6982bb8ff14dcfbb6769b293638293:
>
>   libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
>
> Frederic Martinsons (3):
>   cargo.bbclass: Use --frozen flag for cargo operations
>   cargo_common.bbclass: Handle Cargo.lock modifications for git
>     dependencies
>   cargo.bbclass: Offer a way to use --offline instead of --frozen
>
>  meta/classes-recipe/cargo.bbclass             |  9 +++-
>  meta/classes-recipe/cargo_common.bbclass      | 43 +++++++++++++++++++
>  .../rust-example/rust-hello-world_git.bb      |  2 +
>  3 files changed, 53 insertions(+), 1 deletion(-)
>
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#185128): https://lists.openembedded.org/g/openembedded-core/message/185128
> Mute This Topic: https://lists.openembedded.org/mt/100458213/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Frédéric Martinsons Aug. 1, 2023, 9:09 a.m. UTC | #2
Hello Alexander,

I tried to explain in the code and commit comments, you may also want to
read the ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15104
and I try to explain and ask for help in a dedicated topic on this very
list: see https://lists.openembedded.org/g/openembedded-core/topic/100254129

But to summarize, Cargo.lock is required for various purposes and it
includes the patching path process to work.
I think having no Cargo.lock for a rust recipe built under yocto is not
desirable.

For the  --frozen flag, it put harder constraints than --offline (--offline
prevent network access but doesn't prevent modification
of Cargo.lock)

The special case to revert the behavior introduced was made to comply with
rust-hello-world existence. I would like
to suppress this recipe instead of having this patch but I don't know if it
is desirable and I'd like to think
for other points of views.


On Mon, 31 Jul 2023 at 19:48, Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> I think you need to better describe the benefits. What are the
> improvements? Can we just stay with --offline?
>
> There's lots of intricate code added around cargo.lock handling, with
> special casing to revert to previous behavior via new variable, but
> I'm struggling to understand: to what end?
>
> Alex
>
>
>
>
>
> On Mon, 31 Jul 2023 at 11:44, Frederic Martinsons
> <frederic.martinsons@gmail.com> wrote:
> >
> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
> >
> > This patch series force the usage of --frozen instead of --offline
> during cargo build.
> > This has the advantage to be sure that Cargo.lock file will not be
> modified.
> > Moreover, raise a clear error when Cargo.lock is not present.
> > For this to work, we must modify ourself the Cargo.lock before building
> > in case there are git repo that have been patched.
> >
> > The last commit is to handle the specific case of rust-hello-world
> recipe.
> > If this recipe is no more necessary, we can drop this last patch.
> >
> > The following changes since commit
> 9b5b850d6a6982bb8ff14dcfbb6769b293638293:
> >
> >   libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/fmartinsons/openembedded-core
> 15104-check-cargo-lock
> >
> > Frederic Martinsons (3):
> >   cargo.bbclass: Use --frozen flag for cargo operations
> >   cargo_common.bbclass: Handle Cargo.lock modifications for git
> >     dependencies
> >   cargo.bbclass: Offer a way to use --offline instead of --frozen
> >
> >  meta/classes-recipe/cargo.bbclass             |  9 +++-
> >  meta/classes-recipe/cargo_common.bbclass      | 43 +++++++++++++++++++
> >  .../rust-example/rust-hello-world_git.bb      |  2 +
> >  3 files changed, 53 insertions(+), 1 deletion(-)
> >
> > --
> > 2.34.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#185128):
> https://lists.openembedded.org/g/openembedded-core/message/185128
> > Mute This Topic: https://lists.openembedded.org/mt/100458213/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
Alexander Kanavin Aug. 1, 2023, 9:42 a.m. UTC | #3
Thanks for explaining, I suppose you can simply rework the patchset to
remove rust-hello-world, and rework all places where it's used (you
can grep the poky source tree :)

Alex

On Tue, 1 Aug 2023 at 11:09, Frédéric Martinsons
<frederic.martinsons@gmail.com> wrote:
>
> Hello Alexander,
>
> I tried to explain in the code and commit comments, you may also want to read the ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15104
> and I try to explain and ask for help in a dedicated topic on this very list: see https://lists.openembedded.org/g/openembedded-core/topic/100254129
>
> But to summarize, Cargo.lock is required for various purposes and it includes the patching path process to work.
> I think having no Cargo.lock for a rust recipe built under yocto is not desirable.
>
> For the  --frozen flag, it put harder constraints than --offline (--offline prevent network access but doesn't prevent modification
> of Cargo.lock)
>
> The special case to revert the behavior introduced was made to comply with rust-hello-world existence. I would like
> to suppress this recipe instead of having this patch but I don't know if it is desirable and I'd like to think
> for other points of views.
>
>
> On Mon, 31 Jul 2023 at 19:48, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>>
>> I think you need to better describe the benefits. What are the
>> improvements? Can we just stay with --offline?
>>
>> There's lots of intricate code added around cargo.lock handling, with
>> special casing to revert to previous behavior via new variable, but
>> I'm struggling to understand: to what end?
>>
>> Alex
>>
>>
>>
>>
>>
>> On Mon, 31 Jul 2023 at 11:44, Frederic Martinsons
>> <frederic.martinsons@gmail.com> wrote:
>> >
>> > From: Frederic Martinsons <frederic.martinsons@gmail.com>
>> >
>> > This patch series force the usage of --frozen instead of --offline during cargo build.
>> > This has the advantage to be sure that Cargo.lock file will not be modified.
>> > Moreover, raise a clear error when Cargo.lock is not present.
>> > For this to work, we must modify ourself the Cargo.lock before building
>> > in case there are git repo that have been patched.
>> >
>> > The last commit is to handle the specific case of rust-hello-world recipe.
>> > If this recipe is no more necessary, we can drop this last patch.
>> >
>> > The following changes since commit 9b5b850d6a6982bb8ff14dcfbb6769b293638293:
>> >
>> >   libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
>> >
>> > are available in the Git repository at:
>> >
>> >   https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
>> >
>> > Frederic Martinsons (3):
>> >   cargo.bbclass: Use --frozen flag for cargo operations
>> >   cargo_common.bbclass: Handle Cargo.lock modifications for git
>> >     dependencies
>> >   cargo.bbclass: Offer a way to use --offline instead of --frozen
>> >
>> >  meta/classes-recipe/cargo.bbclass             |  9 +++-
>> >  meta/classes-recipe/cargo_common.bbclass      | 43 +++++++++++++++++++
>> >  .../rust-example/rust-hello-world_git.bb      |  2 +
>> >  3 files changed, 53 insertions(+), 1 deletion(-)
>> >
>> > --
>> > 2.34.1
>> >
>> >
>> > -=-=-=-=-=-=-=-=-=-=-=-
>> > Links: You receive all messages sent to this group.
>> > View/Reply Online (#185128): https://lists.openembedded.org/g/openembedded-core/message/185128
>> > Mute This Topic: https://lists.openembedded.org/mt/100458213/1686489
>> > Group Owner: openembedded-core+owner@lists.openembedded.org
>> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
>> > -=-=-=-=-=-=-=-=-=-=-=-
>> >