From patchwork Fri Dec 15 09:38:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alassane Yattara X-Patchwork-Id: 36363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CFF7C46CCD for ; Fri, 15 Dec 2023 09:39:18 +0000 (UTC) Received: from mail.savoirfairelinux.com (mail.savoirfairelinux.com [208.88.110.44]) by mx.groups.io with SMTP id smtpd.web11.59364.1702633151830413534 for ; Fri, 15 Dec 2023 01:39:12 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@savoirfairelinux.com header.s=DFC430D2-D198-11EC-948E-34200CB392D2 header.b=aEjEdylh; spf=pass (domain: savoirfairelinux.com, ip: 208.88.110.44, mailfrom: alassane.yattara@savoirfairelinux.com) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 394A99C33C0 for ; Fri, 15 Dec 2023 04:39:11 -0500 (EST) Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10032) with ESMTP id ENurVbfG2Ici; Fri, 15 Dec 2023 04:39:10 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.savoirfairelinux.com (Postfix) with ESMTP id 332369C117B; Fri, 15 Dec 2023 04:39:10 -0500 (EST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.savoirfairelinux.com 332369C117B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=savoirfairelinux.com; s=DFC430D2-D198-11EC-948E-34200CB392D2; t=1702633150; bh=7CrJoJDAPm72Y22bplJ8cy9CCOB4NwW8awAHi54WGXk=; h=From:To:Date:Message-Id:MIME-Version; b=aEjEdylhbHceCZYu+5pZ5rqlVSsv/6JFvQ2qI6ikk0fH0e6HALylaccdLW6dt86Rf fMprLQvWYRH8JxvgZjHgl5uEWb/x/7kS338KBgJ/Wr4inj8SUUdYpxw3dpcrkGPJHr oZztTlMv8x6cSNvuLz21bgYjxcl2XMGeew6BTBsFVNNgh8qh+5BmiHwQ6OSexvEQsX QlszzMZ7REPrbYc3MRanLw3w9eFyxrHi5AfrzOzQumHDK96x+WWxNoMMJd7cOGzEvC V2MFkn/X6XZCqu72IWfjyScSPnqG+zuTilSVNV4vPAkcxZXVTqOQTFGojHpv6j654x 9G1KXg2sUhZ7w== X-Virus-Scanned: amavis at mail.savoirfairelinux.com Received: from mail.savoirfairelinux.com ([127.0.0.1]) by localhost (mail.savoirfairelinux.com [127.0.0.1]) (amavis, port 10026) with ESMTP id 4TacV8Nk4Gmh; Fri, 15 Dec 2023 04:39:10 -0500 (EST) Received: from jedi.. (unknown [196.127.183.75]) by mail.savoirfairelinux.com (Postfix) with ESMTPSA id 495309C33C0; Fri, 15 Dec 2023 04:39:09 -0500 (EST) From: Alassane Yattara To: bitbake-devel@lists.openembedded.org Cc: Alassane Yattara Subject: [PATCH v3 5/5] toaster/test: Bug-Fix testcase from bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py Date: Fri, 15 Dec 2023 10:38:39 +0100 Message-Id: <20231215093839.99358-5-alassane.yattara@savoirfairelinux.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231215093839.99358-1-alassane.yattara@savoirfairelinux.com> References: <20231215093839.99358-1-alassane.yattara@savoirfairelinux.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 15 Dec 2023 09:39:18 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/15686 All issues and failures stemmed from a specific test case: test_project_config_tab_right_section in the file bitbake/lib/toaster/tests/functional/test_project_page_tab_config.py. This test was designed to verify whether the "Most built recipes" section on the project page correctly displays the latest and oldest recipes built by the user, irrespective of the build outcome (failed, cancelled, succeeded, or errored). The errors and failures arose because the build process did not terminate as expected, particularly when attempting to build recipe images such as "core-image-minimal" or "bash." It was discovered that building a real recipe/image was unnecessary for the test's purpose. Instead, building a fake recipe like "foo" provided a reliable way to ensure the build would fail or be interrupted. Signed-off-by: Alassane Yattara --- .../tests/functional/test_project_page.py | 2 +- .../test_project_page_tab_config.py | 186 +++++++++--------- 2 files changed, 95 insertions(+), 93 deletions(-) diff --git a/lib/toaster/tests/functional/test_project_page.py b/lib/toaster/tests/functional/test_project_page.py index 077badb0..82dca442 100644 --- a/lib/toaster/tests/functional/test_project_page.py +++ b/lib/toaster/tests/functional/test_project_page.py @@ -461,7 +461,7 @@ class TestProjectPage(SeleniumFunctionalTestCase): '//td[@class="add-del-layers"]//a[1]' ) build_btn.click() - build_state = wait_until_build(self, 'parsing starting cloning queued') + build_state = wait_until_build(self, 'queued cloning starting parsing failed') lastest_builds = self.driver.find_elements( By.XPATH, '//div[@id="latest-builds"]/div' diff --git a/lib/toaster/tests/functional/test_project_page_tab_config.py b/lib/toaster/tests/functional/test_project_page_tab_config.py index d911ff00..4dbf5aeb 100644 --- a/lib/toaster/tests/functional/test_project_page_tab_config.py +++ b/lib/toaster/tests/functional/test_project_page_tab_config.py @@ -12,7 +12,7 @@ import pytest from django.urls import reverse from selenium.webdriver import Keys from selenium.webdriver.support.select import Select -from selenium.common.exceptions import TimeoutException +from selenium.common.exceptions import NoSuchElementException, TimeoutException from orm.models import Project from tests.functional.functional_helpers import SeleniumFunctionalTestCase from selenium.webdriver.common.by import By @@ -26,17 +26,18 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): PROJECT_NAME = 'TestProjectConfigTab' project_id = None - def _create_project(self, project_name): + def _create_project(self, project_name, **kwargs): """ Create/Test new project using: - Project Name: Any string - Release: Any string - Merge Toaster settings: True or False """ + release = kwargs.get('release', '3') self.get(reverse('newproject')) self.wait_until_visible('#new-project-name') self.find("#new-project-name").send_keys(project_name) select = Select(self.find("#projectversion")) - select.select_by_value('3') + select.select_by_value(release) # check merge toaster settings checkbox = self.find('.checkbox-mergeattr') @@ -50,7 +51,7 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): self.find("#create-project-button").click() try: - self.wait_until_visible('#hint-error-project-name') + self.wait_until_visible('#hint-error-project-name', poll=3) url = reverse('project', args=(TestProjectConfigTab.project_id, )) self.get(url) self.wait_until_visible('#config-nav', poll=3) @@ -67,7 +68,8 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): if TestProjectConfigTab.project_id is None: self._create_project(project_name=self._random_string(10)) current_url = self.driver.current_url - TestProjectConfigTab.project_id = get_projectId_from_url(current_url) + TestProjectConfigTab.project_id = get_projectId_from_url( + current_url) else: url = reverse('project', args=(TestProjectConfigTab.project_id,)) self.get(url) @@ -76,27 +78,30 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): def _create_builds(self): # check search box can be use to build recipes search_box = self.find('#build-input') - search_box.send_keys('core-image-minimal') + search_box.send_keys('foo') self.find('#build-button').click() - self.wait_until_visible('#latest-builds') + self.wait_until_present('#latest-builds') # loop until reach the parsing state - build_state = wait_until_build(self, 'parsing starting cloning') + wait_until_build(self, 'queued cloning starting parsing failed') lastest_builds = self.driver.find_elements( By.XPATH, '//div[@id="latest-builds"]/div', ) last_build = lastest_builds[0] self.assertTrue( - 'core-image-minimal' in str(last_build.text) + 'foo' in str(last_build.text) ) - cancel_button = last_build.find_element( - By.XPATH, - '//span[@class="cancel-build-btn pull-right alert-link"]', - ) - cancel_button.click() - if 'starting' not in build_state: # change build state when cancelled in starting state - wait_until_build_cancelled(self) - return build_state + last_build = lastest_builds[0] + try: + cancel_button = last_build.find_element( + By.XPATH, + '//span[@class="cancel-build-btn pull-right alert-link"]', + ) + cancel_button.click() + except NoSuchElementException: + # Skip if the build is already cancelled + pass + wait_until_build_cancelled(self) def _get_tabs(self): # tabs links list @@ -126,6 +131,7 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): - Delete project """ self._navigate_to_project_page() + def _get_config_nav_item(index): config_nav = self.find('#config-nav') return config_nav.find_elements(By.TAG_NAME, 'li')[index] @@ -152,13 +158,27 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): self.assertTrue("actions" in str(actions.text).lower()) conf_nav_list = [ - [0, 'Configuration', f"/toastergui/project/{TestProjectConfigTab.project_id}"], # config - [2, 'Custom images', f"/toastergui/project/{TestProjectConfigTab.project_id}/customimages"], # custom images - [3, 'Image recipes', f"/toastergui/project/{TestProjectConfigTab.project_id}/images"], # image recipes - [4, 'Software recipes', f"/toastergui/project/{TestProjectConfigTab.project_id}/softwarerecipes"], # software recipes - [5, 'Machines', f"/toastergui/project/{TestProjectConfigTab.project_id}/machines"], # machines - [6, 'Layers', f"/toastergui/project/{TestProjectConfigTab.project_id}/layers"], # layers - [7, 'Distros', f"/toastergui/project/{TestProjectConfigTab.project_id}/distros"], # distro + # config + [0, 'Configuration', + f"/toastergui/project/{TestProjectConfigTab.project_id}"], + # custom images + [2, 'Custom images', + f"/toastergui/project/{TestProjectConfigTab.project_id}/customimages"], + # image recipes + [3, 'Image recipes', + f"/toastergui/project/{TestProjectConfigTab.project_id}/images"], + # software recipes + [4, 'Software recipes', + f"/toastergui/project/{TestProjectConfigTab.project_id}/softwarerecipes"], + # machines + [5, 'Machines', + f"/toastergui/project/{TestProjectConfigTab.project_id}/machines"], + # layers + [6, 'Layers', + f"/toastergui/project/{TestProjectConfigTab.project_id}/layers"], + # distro + [7, 'Distros', + f"/toastergui/project/{TestProjectConfigTab.project_id}/distros"], # [9, 'BitBake variables', f"/toastergui/project/{TestProjectConfigTab.project_id}/configuration"], # bitbake variables ] for index, item_name, url in conf_nav_list: @@ -281,15 +301,10 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): # Create a new project for this test project_name = self._random_string(10) self._create_project(project_name=project_name) - current_url = self.driver.current_url - TestProjectConfigTab.project_id = get_projectId_from_url(current_url) - url = current_url.split('?')[0] # check if the menu is displayed self.wait_until_visible('#project-page') block_l = self.driver.find_element( By.XPATH, '//*[@id="project-page"]/div[2]') - most_built_recipes = self.driver.find_element( - By.XPATH, '//*[@id="project-page"]/div[1]/div[3]') project_release = self.driver.find_element( By.XPATH, '//*[@id="project-page"]/div[1]/div[4]') layers = block_l.find_element(By.ID, 'layer-container') @@ -315,26 +330,6 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): f'You have changed the {item_name} to: {new_item_name}' in change_notification.text ) - def rebuild_from_most_build_recipes(recipe_list_items): - checkbox = recipe_list_items[0].find_element(By.TAG_NAME, 'input') - checkbox.click() - build_btn = self.find('#freq-build-btn') - build_btn.click() - self.wait_until_visible('#latest-builds') - build_state = wait_until_build(self, 'parsing starting cloning queued') - lastest_builds = self.driver.find_elements( - By.XPATH, - '//div[@id="latest-builds"]/div' - ) - last_build = lastest_builds[0] - self.assertTrue(len(lastest_builds) >= 2) - cancel_button = last_build.find_element( - By.XPATH, - '//span[@class="cancel-build-btn pull-right alert-link"]', - ) - cancel_button.click() - if 'starting' not in build_state: # change build state when cancelled in starting state - wait_until_build_cancelled(self) # Machine check_machine_distro(self, 'machine', 'qemux86-64', 'machine-section') # Distro @@ -374,32 +369,61 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): layers_list_items = layers_list.find_elements(By.TAG_NAME, 'li') self.assertTrue(len(layers_list_items) == 4) - # Most built recipes - title = most_built_recipes.find_element(By.TAG_NAME, 'h3') - self.assertTrue("Most built recipes" in title.text) + def test_most_build_recipes(self): + """ Test most build recipes block contains""" + def rebuild_from_most_build_recipes(recipe_list_items): + checkbox = recipe_list_items[0].find_element(By.TAG_NAME, 'input') + checkbox.click() + build_btn = self.find('#freq-build-btn') + build_btn.click() + self.wait_until_present('#latest-builds') + wait_until_build(self, 'queued cloning starting parsing failed') + lastest_builds = self.driver.find_elements( + By.XPATH, + '//div[@id="latest-builds"]/div' + ) + self.assertTrue(len(lastest_builds) >= 2) + last_build = lastest_builds[0] + try: + cancel_button = last_build.find_element( + By.XPATH, + '//span[@class="cancel-build-btn pull-right alert-link"]', + ) + cancel_button.click() + except NoSuchElementException: + # Skip if the build is already cancelled + pass + wait_until_build_cancelled(self) + # Create a new project for remaining asserts + project_name = self._random_string(10) + self._create_project(project_name=project_name, release='2') + current_url = self.driver.current_url + TestProjectConfigTab.project_id = get_projectId_from_url(current_url) + url = current_url.split('?')[0] + # Create a new builds - build_state = self._create_builds() + self._create_builds() - # Refresh the page + # back to project page self.driver.get(url) self.wait_until_visible('#project-page', poll=3) - # check can select a recipe and build it + + # Most built recipes most_built_recipes = self.driver.find_element( By.XPATH, '//*[@id="project-page"]/div[1]/div[3]') - recipe_list = most_built_recipes.find_element(By.ID, 'freq-build-list') + title = most_built_recipes.find_element(By.TAG_NAME, 'h3') + self.assertTrue("Most built recipes" in title.text) + # check can select a recipe and build it + self.wait_until_visible('#freq-build-list', poll=3) + recipe_list = self.find('#freq-build-list') recipe_list_items = recipe_list.find_elements(By.TAG_NAME, 'li') - if 'starting' not in build_state: # Build will not appear in the list if canceled in starting state - self.assertTrue( - len(recipe_list_items) > 0, - msg="No recipes found in the most built recipes list", - ) - rebuild_from_most_build_recipes(recipe_list_items) - else: - self.assertTrue( - len(recipe_list_items) == 0, - msg="Recipes found in the most built recipes list", - ) + self.assertTrue( + len(recipe_list_items) > 0, + msg="Any recipes found in the most built recipes list", + ) + rebuild_from_most_build_recipes(recipe_list_items) + TestProjectConfigTab.project_id = None # reset project id def test_project_page_tab_importlayer(self): """ Test project page tab import layer """ @@ -461,10 +485,9 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): div_empty_msg = self.find('#empty-state-customimagestable') link_create_custom_image = div_empty_msg.find_element( By.TAG_NAME, 'a') - last_project_id = Project.objects.get(name=project_name).id - self.assertTrue(last_project_id is not None) + self.assertTrue(TestProjectConfigTab.project_id is not None) self.assertTrue( - f"/toastergui/project/{last_project_id}/newcustomimage" in str( + f"/toastergui/project/{TestProjectConfigTab.project_id}/newcustomimage" in str( link_create_custom_image.get_attribute('href') ) ) @@ -473,6 +496,7 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): link_create_custom_image.text ) ) + TestProjectConfigTab.project_id = None # reset project id def test_project_page_image_recipe(self): """ Test project page section images @@ -497,25 +521,3 @@ class TestProjectConfigTab(SeleniumFunctionalTestCase): self.wait_until_visible('#imagerecipestable tbody tr') rows = self.find_all('#imagerecipestable tbody tr') self.assertTrue(len(rows) > 0) - - # Test build button - image_to_build = rows[0] - build_btn = image_to_build.find_element( - By.XPATH, - '//td[@class="add-del-layers"]' - ) - build_btn.click() - build_state = wait_until_build(self, 'parsing starting cloning queued') - lastest_builds = self.driver.find_elements( - By.XPATH, - '//div[@id="latest-builds"]/div' - ) - self.assertTrue(len(lastest_builds) > 0) - last_build = lastest_builds[0] - cancel_button = last_build.find_element( - By.XPATH, - '//span[@class="cancel-build-btn pull-right alert-link"]', - ) - cancel_button.click() - if 'starting' not in build_state: # change build state when cancelled in starting state - wait_until_build_cancelled(self)