toaster: large package set breaks sqlite query

Submitted by Reyna, David on June 24, 2017, 5:18 a.m. | Patch ID: 141229

Details

Message ID 1498281525-4390-1-git-send-email-david.reyna@windriver.com
State New
Headers show

Commit Message

Reyna, David June 24, 2017, 5:18 a.m. 
From: David Reyna <David.Reyna@windriver.com>

If you build a project with a large package set, you will get a crash
in "views.py" when the dashboard attempts to fetch the package set to
calculate the package count and size. This is a sqlite limitation, and
it fails with as few as 1220 packages.

[YOCTO #11717]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 bitbake/lib/toaster/toastergui/views.py | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

Patch hide | download patch | download mbox 

diff --git a/bitbake/lib/toaster/toastergui/views.py b/bitbake/lib/toaster/toastergui/views.py
index 75c5911..5720b9d 100755
--- a/bitbake/lib/toaster/toastergui/views.py
+++ b/bitbake/lib/toaster/toastergui/views.py
@@ -457,10 +457,15 @@  def builddashboard( request, build_id ):
         npkg = 0
         pkgsz = 0
         package = None
-        for package in Package.objects.filter(id__in = [x.package_id for x in t.target_installed_package_set.all()]):
-            pkgsz = pkgsz + package.size
-            if package.installed_name:
-                npkg = npkg + 1
+        # Chunk the query to avoid "too many SQL variables" error
+        package_set = t.target_installed_package_set.all()
+        package_set_len = len(package_set)
+        for ps_start in range(0,package_set_len,500):
+            ps_stop = min(ps_start+500,package_set_len)
+            for package in Package.objects.filter(id__in = [x.package_id for x in package_set[ps_start:ps_stop]]):
+                pkgsz = pkgsz + package.size
+                if package.installed_name:
+                    npkg = npkg + 1
         elem['npkg'] = npkg
         elem['pkgsz'] = pkgsz
         ti = Target_Image_File.objects.filter(target_id = t.id)