Patchwork [2/2] rootfs_rpm.bbclass: don't remove rpmlib when INC_RPM_IMAGE_GEN=1

login
register
mail settings
Submitter Robert Yang
Date May 9, 2012, 8:20 a.m.
Message ID <03d9cdca29477f34348a0f97138504441c5804fa.1336547713.git.liezhi.yang@windriver.com>
Download mbox | patch
Permalink /patch/27373/
State New
Headers show

Comments

Robert Yang - May 9, 2012, 8:20 a.m.
Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
list_installed_packages would get nothing in the second build.

[YOCTO #2440]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/rootfs_rpm.bbclass |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)
Koen Kooi - May 9, 2012, 8:50 a.m.
Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:

> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
> list_installed_packages would get nothing in the second build.

What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?
Mark Hatle - May 9, 2012, 2:52 p.m.
On 5/9/12 3:50 AM, Koen Kooi wrote:
>
> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>
>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>> list_installed_packages would get nothing in the second build.
>
> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?

The intention is that only the packages that have been changed/upgraded will be 
installed.  Instead of the whole image being generated from scratch each time.

--Mark
Koen Kooi - May 9, 2012, 3:01 p.m.
Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:

> On 5/9/12 3:50 AM, Koen Kooi wrote:
>> 
>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>> 
>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>> list_installed_packages would get nothing in the second build.
>> 
>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?
> 
> The intention is that only the packages that have been changed/upgraded will be installed.  Instead of the whole image being generated from scratch each time.

That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this patch addresses the sequence I outlined above.
Mark Hatle - May 9, 2012, 3:15 p.m.
On 5/9/12 10:01 AM, Koen Kooi wrote:
>
> Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:
>
>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>
>>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>>>
>>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>>> list_installed_packages would get nothing in the second build.
>>>
>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?
>>
>> The intention is that only the packages that have been changed/upgraded will be installed.  Instead of the whole image being generated from scratch each time.
>
> That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this patch addresses the sequence I outlined above.

Explain what you mean by "build".

Using the following:

. ./oe-init-build-env build-test
<setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
bitbake core-image-core
--- time passes, git pull pulls in changes ---
bitbake core-image-core

The second "build", only the changed packages will be added to the image.

You you are using different build directories, then it doesn't re-use anything. 
  The purpose of this is incremental software development within a single build 
directory.

--Mark

> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Koen Kooi - May 9, 2012, 3:24 p.m.
Op 9 mei 2012, om 17:15 heeft Mark Hatle het volgende geschreven:

> On 5/9/12 10:01 AM, Koen Kooi wrote:
>> 
>> Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:
>> 
>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>> 
>>>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>>>> 
>>>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>>>> list_installed_packages would get nothing in the second build.
>>>> 
>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?
>>> 
>>> The intention is that only the packages that have been changed/upgraded will be installed.  Instead of the whole image being generated from scratch each time.
>> 
>> That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this patch addresses the sequence I outlined above.
> 
> Explain what you mean by "build".
> 
> Using the following:
> 
> . ./oe-init-build-env build-test
> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
> bitbake core-image-core
> --- time passes, git pull pulls in changes ---
> bitbake core-image-core
> 
> The second "build", only the changed packages will be added to the image.
> 
> You you are using different build directories, then it doesn't re-use anything.  The purpose of this is incremental software development within a single build directory.

You are still missing my point. The patch is broken if you change INC_RPM_IMAGE_GEN
Mark Hatle - May 9, 2012, 3:34 p.m.
On 5/9/12 10:24 AM, Koen Kooi wrote:
>
> Op 9 mei 2012, om 17:15 heeft Mark Hatle het volgende geschreven:
>
>> On 5/9/12 10:01 AM, Koen Kooi wrote:
>>>
>>> Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:
>>>
>>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>>>
>>>>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>>>>>
>>>>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>>>>> list_installed_packages would get nothing in the second build.
>>>>>
>>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?
>>>>
>>>> The intention is that only the packages that have been changed/upgraded will be installed.  Instead of the whole image being generated from scratch each time.
>>>
>>> That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this patch addresses the sequence I outlined above.
>>
>> Explain what you mean by "build".
>>
>> Using the following:
>>
>> . ./oe-init-build-env build-test
>> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
>> bitbake core-image-core
>> --- time passes, git pull pulls in changes ---
>> bitbake core-image-core
>>
>> The second "build", only the changed packages will be added to the image.
>>
>> You you are using different build directories, then it doesn't re-use anything.  The purpose of this is incremental software development within a single build directory.
>
> You are still missing my point. The patch is broken if you change INC_RPM_IMAGE_GEN

I am absolutely missing your point.

I don't see what is wrong in the following, but I haven't tried it:

> -	rm -rf ${IMAGE_ROOTFS}${rpmlibdir}
> -	rm -rf ${IMAGE_ROOTFS}${opkglibdir}
> +	# Don't remove the rpmlib when INC_RPM_IMAGE_GEN=1
> +	if [ "${INC_RPM_IMAGE_GEN}" != "1" ]; then
> +		rm -rf ${IMAGE_ROOTFS}${rpmlibdir}
> +		rm -rf ${IMAGE_ROOTFS}${opkglibdir}
> +	fi

If the INC_RPM_IMAGE_GEN is not set to '1'.  (Specifically '1' as opposed to 
just checking if it's set.) then it'll remove the existing image packaging database.

--Mark

> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Marko Lindqvist - May 9, 2012, 3:54 p.m.
On 9 May 2012 18:15, Mark Hatle <mark.hatle@windriver.com> wrote:
>>
>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>>
>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another
>>>> build?
>
> Using the following:
>
> . ./oe-init-build-env build-test
> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
> bitbake core-image-core
> --- time passes, git pull pulls in changes ---
> bitbake core-image-core

 That's not the same sequence. You were supposed to make first build
with INC_RPM_IMAGE_GEN unset, and set it for second build only.


 - ML
Mark Hatle - May 9, 2012, 4:04 p.m.
On 5/9/12 10:54 AM, Marko Lindqvist wrote:
> On 9 May 2012 18:15, Mark Hatle<mark.hatle@windriver.com>  wrote:
>>>
>>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>>>
>>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another
>>>>> build?
>>
>> Using the following:
>>
>> . ./oe-init-build-env build-test
>> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
>> bitbake core-image-core
>> --- time passes, git pull pulls in changes ---
>> bitbake core-image-core
>
>   That's not the same sequence. You were supposed to make first build
> with INC_RPM_IMAGE_GEN unset, and set it for second build only.

If the value changes from one build to the next the behavior is undefined.  It 
will probably work, but it's not guaranteed.  (I'd need to look at the full 
implementation to verify that...)

--Mark

>
>   - ML
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Robert Yang - May 10, 2012, 2:13 a.m.
On 05/09/2012 11:24 PM, Koen Kooi wrote:
>
> Op 9 mei 2012, om 17:15 heeft Mark Hatle het volgende geschreven:
>
>> On 5/9/12 10:01 AM, Koen Kooi wrote:
>>>
>>> Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:
>>>
>>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>>>
>>>>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>>>>>
>>>>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>>>>> list_installed_packages would get nothing in the second build.
>>>>>
>>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do another build?
>>>>
>>>> The intention is that only the packages that have been changed/upgraded will be installed.  Instead of the whole image being generated from scratch each time.
>>>
>>> That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this patch addresses the sequence I outlined above.
>>
>> Explain what you mean by "build".
>>
>> Using the following:
>>
>> . ./oe-init-build-env build-test
>> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
>> bitbake core-image-core
>> --- time passes, git pull pulls in changes ---
>> bitbake core-image-core
>>
>> The second "build", only the changed packages will be added to the image.
>>
>> You you are using different build directories, then it doesn't re-use anything.  The purpose of this is incremental software development within a single build directory.
>
> You are still missing my point. The patch is broken if you change INC_RPM_IMAGE_GEN

Only the core-image-minimal has this problem, other images won't remove the
database, the core-image-minimal removes the database to save disk space (
only saves quite a little), and I don't know whether there is a better solution
for the core-image-minimal, maybe move the database to ${T} rather than remove
it, and move it back to the rootfs directory at the early stage of do_rootfs, 
but I don't know whether this is worth or not since this is just used for
saving the time at the development stage, as it had told:

# Incremental rpm image generation, the rootfs would be totally removed
# and re-created in the second generation by default, but with
# INC_RPM_IMAGE_GEN = "1", the rpm based rootfs would be kept, and will
# do update(remove/add some pkgs) on it.  NOTE: This is not suggested
# when you want to create a productive rootfs

// Robert

> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Saul Wold - May 11, 2012, 4:54 p.m.
On 05/09/2012 07:13 PM, Robert Yang wrote:
>
>
> On 05/09/2012 11:24 PM, Koen Kooi wrote:
>>
>> Op 9 mei 2012, om 17:15 heeft Mark Hatle het volgende geschreven:
>>
>>> On 5/9/12 10:01 AM, Koen Kooi wrote:
>>>>
>>>> Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:
>>>>
>>>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>>>>
>>>>>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>>>>>>
>>>>>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>>>>>> list_installed_packages would get nothing in the second build.
>>>>>>
>>>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do
>>>>>> another build?
>>>>>
>>>>> The intention is that only the packages that have been
>>>>> changed/upgraded will be installed. Instead of the whole image
>>>>> being generated from scratch each time.
>>>>
>>>> That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this
>>>> patch addresses the sequence I outlined above.
>>>
>>> Explain what you mean by "build".
>>>
>>> Using the following:
>>>
>>> . ./oe-init-build-env build-test
>>> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
>>> bitbake core-image-core
>>> --- time passes, git pull pulls in changes ---
>>> bitbake core-image-core
>>>
>>> The second "build", only the changed packages will be added to the
>>> image.
>>>
>>> You you are using different build directories, then it doesn't re-use
>>> anything. The purpose of this is incremental software development
>>> within a single build directory.
>>
>> You are still missing my point. The patch is broken if you change
>> INC_RPM_IMAGE_GEN
>
> Only the core-image-minimal has this problem, other images won't remove the
> database, the core-image-minimal removes the database to save disk space (
> only saves quite a little), and I don't know whether there is a better
> solution
> for the core-image-minimal, maybe move the database to ${T} rather than
> remove
> it, and move it back to the rootfs directory at the early stage of
> do_rootfs, but I don't know whether this is worth or not since this is
> just used for
> saving the time at the development stage, as it had told:
>
Robert,

The move might be the best solution since it will preserve it and make 
the behavior more predictable, ie if one starts with INC_RPM_IMAGE_GEN 
disabled and then enables it the package database will be available 
already.  It would also preserve the space in the core-image-minimal by 
not keeping those database files in the image.

Please respin this patch.

Thanks
	Sau!



> # Incremental rpm image generation, the rootfs would be totally removed
> # and re-created in the second generation by default, but with
> # INC_RPM_IMAGE_GEN = "1", the rpm based rootfs would be kept, and will
> # do update(remove/add some pkgs) on it. NOTE: This is not suggested
> # when you want to create a productive rootfs
>
> // Robert
>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
Robert Yang - May 12, 2012, 9:40 a.m.
On 05/12/2012 12:54 AM, Saul Wold wrote:
> On 05/09/2012 07:13 PM, Robert Yang wrote:
>>
>>
>> On 05/09/2012 11:24 PM, Koen Kooi wrote:
>>>
>>> Op 9 mei 2012, om 17:15 heeft Mark Hatle het volgende geschreven:
>>>
>>>> On 5/9/12 10:01 AM, Koen Kooi wrote:
>>>>>
>>>>> Op 9 mei 2012, om 16:52 heeft Mark Hatle het volgende geschreven:
>>>>>
>>>>>> On 5/9/12 3:50 AM, Koen Kooi wrote:
>>>>>>>
>>>>>>> Op 9 mei 2012, om 10:20 heeft Robert Yang het volgende geschreven:
>>>>>>>
>>>>>>>> Don't remove rpmlib when INC_RPM_IMAGE_GEN=1, otherwise the
>>>>>>>> list_installed_packages would get nothing in the second build.
>>>>>>>
>>>>>>> What happens when I do a build, enable INC_RPM_IMAGE_GEN and do
>>>>>>> another build?
>>>>>>
>>>>>> The intention is that only the packages that have been
>>>>>> changed/upgraded will be installed. Instead of the whole image
>>>>>> being generated from scratch each time.
>>>>>
>>>>> That's what INC_RPM_IMAGE_GEN does, yes. What I'm asking is how this
>>>>> patch addresses the sequence I outlined above.
>>>>
>>>> Explain what you mean by "build".
>>>>
>>>> Using the following:
>>>>
>>>> . ./oe-init-build-env build-test
>>>> <setup local.conf w/ rpm packaging and INC_RPM_IMAGE_GEN>
>>>> bitbake core-image-core
>>>> --- time passes, git pull pulls in changes ---
>>>> bitbake core-image-core
>>>>
>>>> The second "build", only the changed packages will be added to the
>>>> image.
>>>>
>>>> You you are using different build directories, then it doesn't re-use
>>>> anything. The purpose of this is incremental software development
>>>> within a single build directory.
>>>
>>> You are still missing my point. The patch is broken if you change
>>> INC_RPM_IMAGE_GEN
>>
>> Only the core-image-minimal has this problem, other images won't remove the
>> database, the core-image-minimal removes the database to save disk space (
>> only saves quite a little), and I don't know whether there is a better
>> solution
>> for the core-image-minimal, maybe move the database to ${T} rather than
>> remove
>> it, and move it back to the rootfs directory at the early stage of
>> do_rootfs, but I don't know whether this is worth or not since this is
>> just used for
>> saving the time at the development stage, as it had told:
>>
> Robert,
>
> The move might be the best solution since it will preserve it and make the
> behavior more predictable, ie if one starts with INC_RPM_IMAGE_GEN disabled and
> then enables it the package database will be available already. It would also
> preserve the space in the core-image-minimal by not keeping those database files
> in the image.
>

Thanks Saul, I will send a V2 for it.

// Robert

> Please respin this patch.
>
> Thanks
> Sau!
>
>
>
>> # Incremental rpm image generation, the rootfs would be totally removed
>> # and re-created in the second generation by default, but with
>> # INC_RPM_IMAGE_GEN = "1", the rpm based rootfs would be kept, and will
>> # do update(remove/add some pkgs) on it. NOTE: This is not suggested
>> # when you want to create a productive rootfs
>>
>> // Robert
>>
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>>
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>>
>

Patch

diff --git a/meta/classes/rootfs_rpm.bbclass b/meta/classes/rootfs_rpm.bbclass
index 9039b21..8ec03d2 100644
--- a/meta/classes/rootfs_rpm.bbclass
+++ b/meta/classes/rootfs_rpm.bbclass
@@ -154,8 +154,11 @@  EOF
 }
 
 remove_packaging_data_files() {
-	rm -rf ${IMAGE_ROOTFS}${rpmlibdir}
-	rm -rf ${IMAGE_ROOTFS}${opkglibdir}
+	# Don't remove the rpmlib when INC_RPM_IMAGE_GEN=1
+	if [ "${INC_RPM_IMAGE_GEN}" != "1" ]; then
+		rm -rf ${IMAGE_ROOTFS}${rpmlibdir}
+		rm -rf ${IMAGE_ROOTFS}${opkglibdir}
+	fi
 }
 
 RPM_QUERY_CMD = '${RPM} --root ${IMAGE_ROOTFS} -D "_dbpath ${rpmlibdir}" \