| Message ID | 20220107231526.1517563-2-richard.purdie@linuxfoundation.org |
|---|---|
| State | Accepted, archived |
| Commit | 0746b6a2a32fec4c18bf1a52b1454ca4c04bf543 |
| Headers | show |
| Series | [1/2] utils: Add disable_network function | expand |
Hi RP, On 1/8/22 7:15 AM, Richard Purdie wrote: > This patch changes behaviour. After this change any task which does not > have the network flag set will have networking disabled on systems that > support that (kernel version dependent). > > Add a "network" task specific flag which then triggers networking to > be enabled for this task, it is otherwise disabled. > > This needs to happen before we enter the fakeroot environment of the task > due to the need for the real uid/gid which we save in the parent process. > It seems that this patch doesn't work with NIS user, since NIS user alwasy try to access the network to translate UID to USER, but there is no network, so we will get the errors like the following during the build: do_ypcall: clnt_call: RPC: Unable to send; errno = Network is unreachable These error messages may cause build failures such as 1) bitbake quilt-native -cdevshell doesn't work when set OE_TERMINAL = "screen" in conf/local.conf: ERROR: quilt-native-0.66-r0 do_devshell: Unable to spawn terminal screen: Execution of /path/to/0.66-r0/temp/run.do_terminal.189055' failed with exit code 1: getpwuid() can't identify your account! If no OE_TERMINAL is setting, it will go into a shell with "I have no name!": I have no name!@host:/path/to/quilt-0.66$ 2) bitbake cmake-native failed This because the error string "do_ypcall: clnt_call: RPC: Unable to send; errno = Network is unreachable" has been assigned to a variable, thus caused while errors, for example: VAR=$(whomai) It will get the string "do_ypcall: clnt_call: RPC: Unable to send; errno = Network is unreachable", and cause build errors. // Robert > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > bin/bitbake-worker | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/bin/bitbake-worker b/bin/bitbake-worker > index bf96207edc..3aaf3c2444 100755 > --- a/bin/bitbake-worker > +++ b/bin/bitbake-worker > @@ -152,6 +152,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, fn, task, taskname, taskha > fakeenv = {} > umask = None > > + uid = os.getuid() > + gid = os.getgid() > + > + > taskdep = workerdata["taskdeps"][fn] > if 'umask' in taskdep and taskname in taskdep['umask']: > umask = taskdep['umask'][taskname] > @@ -257,6 +261,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, fn, task, taskname, taskha > > bb.utils.set_process_name("%s:%s" % (the_data.getVar("PN"), taskname.replace("do_", ""))) > > + if not the_data.getVarFlag(taskname, 'network', False): > + logger.debug("Attempting to disable network") > + bb.utils.disable_network(uid, gid) > + > # exported_vars() returns a generator which *cannot* be passed to os.environ.update() > # successfully. We also need to unset anything from the environment which shouldn't be there > exports = bb.data.exported_vars(the_data) > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#13225): https://lists.openembedded.org/g/bitbake-devel/message/13225 > Mute This Topic: https://lists.openembedded.org/mt/88273738/3616940 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [liezhi.yang@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, 2022-01-19 at 19:49 +0800, Robert Yang wrote: > Hi RP, > > On 1/8/22 7:15 AM, Richard Purdie wrote: > > This patch changes behaviour. After this change any task which does not > > have the network flag set will have networking disabled on systems that > > support that (kernel version dependent). > > > > Add a "network" task specific flag which then triggers networking to > > be enabled for this task, it is otherwise disabled. > > > > This needs to happen before we enter the fakeroot environment of the task > > due to the need for the real uid/gid which we save in the parent process. > > > > It seems that this patch doesn't work with NIS user, since NIS user alwasy try > to access the network to translate UID to USER, but there is no network, so we > will get the errors like the following during the build: > > do_ypcall: clnt_call: RPC: Unable to send; errno = Network is unreachable > > These error messages may cause build failures such as > > 1) bitbake quilt-native -cdevshell doesn't work when set OE_TERMINAL = "screen" > in conf/local.conf: > ERROR: quilt-native-0.66-r0 do_devshell: Unable to spawn terminal screen: > Execution of /path/to/0.66-r0/temp/run.do_terminal.189055' failed with exit code 1: > getpwuid() can't identify your account! > > If no OE_TERMINAL is setting, it will go into a shell with "I have no name!": > > I have no name!@host:/path/to/quilt-0.66$ > > 2) bitbake cmake-native failed > This because the error string "do_ypcall: clnt_call: RPC: Unable to send; errno > = Network is unreachable" has been assigned to a variable, thus caused while > errors, for example: > VAR=$(whomai) > > It will get the string "do_ypcall: clnt_call: RPC: Unable to send; errno = > Network is unreachable", and cause build errors. Thanks for the report, that is rather sad/unfortunate. I guess we'll have to add some way to allow network access globally for such machines, much as I'd prefer not to. Cheers, Richard
Hi Richard, Unfortunately this patch breaks the usage of the icecc.bbclass. Currently I build with icecc inside a container with network isolation and my icecc demon runs outside of the container. The only thing I need to do for using the icecc inside my build container is mounting the unix /var/run/icecc/iceccd.socket inside the container. I think that we need something like this functionality to have access to some sockets connections inside the tasks with the new namespace. Jose Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia sexta, 7/01/2022 à(s) 23:15: > This patch changes behaviour. After this change any task which does not > have the network flag set will have networking disabled on systems that > support that (kernel version dependent). > > Add a "network" task specific flag which then triggers networking to > be enabled for this task, it is otherwise disabled. > > This needs to happen before we enter the fakeroot environment of the task > due to the need for the real uid/gid which we save in the parent process. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > bin/bitbake-worker | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/bin/bitbake-worker b/bin/bitbake-worker > index bf96207edc..3aaf3c2444 100755 > --- a/bin/bitbake-worker > +++ b/bin/bitbake-worker > @@ -152,6 +152,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, > fn, task, taskname, taskha > fakeenv = {} > umask = None > > + uid = os.getuid() > + gid = os.getgid() > + > + > taskdep = workerdata["taskdeps"][fn] > if 'umask' in taskdep and taskname in taskdep['umask']: > umask = taskdep['umask'][taskname] > @@ -257,6 +261,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, > fn, task, taskname, taskha > > bb.utils.set_process_name("%s:%s" % > (the_data.getVar("PN"), taskname.replace("do_", ""))) > > + if not the_data.getVarFlag(taskname, 'network', False): > + logger.debug("Attempting to disable network") > + bb.utils.disable_network(uid, gid) > + > # exported_vars() returns a generator which *cannot* be > passed to os.environ.update() > # successfully. We also need to unset anything from the > environment which shouldn't be there > exports = bb.data.exported_vars(the_data) > -- > 2.32.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#13225): > https://lists.openembedded.org/g/bitbake-devel/message/13225 > Mute This Topic: https://lists.openembedded.org/mt/88273738/5052612 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia quarta, 19/01/2022 à(s) 23:27: > On Wed, 2022-01-19 at 23:16 +0000, Jose Quaresma wrote: > > Unfortunately this patch breaks the usage of the icecc.bbclass. > > > > Currently I build with icecc inside a container with network isolation > and > > my icecc demon runs outside of the container. > > The only thing I need to do for using the icecc inside my build > container is > > mounting the unix /var/run/icecc/iceccd.socket inside the container. > > > > I think that we need something like this functionality to have access to > > some sockets connections inside the tasks with the new namespace. > > I'm open to proposals but the unshare functionality is limited so I > suspect we > can't just allow/disallow specific sockets unfortunately. We may just need > the > global "allow" flag for the reasons Robert mentioned which would also work > here, > at the code of not showing up other network accesses. > > Cheers, > > Richard > > I will do some more investigation on that to see how we can use some pre-existing unix sockets from the old namespace. I think that the IPC namespace is used for that purpose but I need to understand more about this subject.
On 1/20/22 7:27 AM, Richard Purdie wrote: > On Wed, 2022-01-19 at 23:16 +0000, Jose Quaresma wrote: >> Unfortunately this patch breaks the usage of the icecc.bbclass. >> >> Currently I build with icecc inside a container with network isolation and >> my icecc demon runs outside of the container. >> The only thing I need to do for using the icecc inside my build container is >> mounting the unix /var/run/icecc/iceccd.socket inside the container. >> >> I think that we need something like this functionality to have access to >> some sockets connections inside the tasks with the new namespace. > > I'm open to proposals but the unshare functionality is limited so I suspect we > can't just allow/disallow specific sockets unfortunately. We may just need the > global "allow" flag for the reasons Robert mentioned which would also work here, > at the code of not showing up other network accesses. I will send 2 patches to make we can global allow network accesses, one is for bitbake, the other one for oe-core. // Robert > > Cheers, > > Richard > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#13249): https://lists.openembedded.org/g/bitbake-devel/message/13249 > Mute This Topic: https://lists.openembedded.org/mt/88273738/3616940 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [liezhi.yang@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/bin/bitbake-worker b/bin/bitbake-worker index bf96207edc..3aaf3c2444 100755 --- a/bin/bitbake-worker +++ b/bin/bitbake-worker @@ -152,6 +152,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, fn, task, taskname, taskha fakeenv = {} umask = None + uid = os.getuid() + gid = os.getgid() + + taskdep = workerdata["taskdeps"][fn] if 'umask' in taskdep and taskname in taskdep['umask']: umask = taskdep['umask'][taskname] @@ -257,6 +261,10 @@ def fork_off_task(cfg, data, databuilder, workerdata, fn, task, taskname, taskha bb.utils.set_process_name("%s:%s" % (the_data.getVar("PN"), taskname.replace("do_", ""))) + if not the_data.getVarFlag(taskname, 'network', False): + logger.debug("Attempting to disable network") + bb.utils.disable_network(uid, gid) + # exported_vars() returns a generator which *cannot* be passed to os.environ.update() # successfully. We also need to unset anything from the environment which shouldn't be there exports = bb.data.exported_vars(the_data)
This patch changes behaviour. After this change any task which does not have the network flag set will have networking disabled on systems that support that (kernel version dependent). Add a "network" task specific flag which then triggers networking to be enabled for this task, it is otherwise disabled. This needs to happen before we enter the fakeroot environment of the task due to the need for the real uid/gid which we save in the parent process. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- bin/bitbake-worker | 8 ++++++++ 1 file changed, 8 insertions(+)