From patchwork Sat Dec 30 04:57:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1340CC46CD2 for ; Sat, 30 Dec 2023 05:00:13 +0000 (UTC) Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by mx.groups.io with SMTP id smtpd.web11.179116.1703912404849945960 for ; Fri, 29 Dec 2023 21:00:04 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=WgNxYJfQ; spf=pass (domain: gmail.com, ip: 209.85.210.175, mailfrom: ticotimo@gmail.com) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6da202aa138so1409748b3a.2 for ; Fri, 29 Dec 2023 21:00:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912404; x=1704517204; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=KfCrAxn+qVVomWZAVcoKxw5/zF1Ne58DSw43xSkBu64=; b=WgNxYJfQW1q+XCGKSMA40NRpzsn/yBJstOff3939cx9nUMhtteb2B1XIK6AwmGEyE+ MZNIfaPOQR9VLq+hjb2NTOe4u89/0cDq4R9u2svfy9B61sXjVGZrK+mjOaqmCopLGkEb 0Mt2JP+fNk9Cp1DiRq+WRr0VZ5yzjTBDYsGv5G+iNVvw4TkGXIpYsN/ziy9IjnC9TaH7 vuOeC2c60N67ovkeR3HcNsVht8ae9+/ou7Gr2/6LLEqz1+5JEMQJI0UJluS3c68vW4+q 6LLkJyroxsw5851YGDA7I2h0kC9C7CzBBD4iokapT1NCaBjyoJBlwFNU4b3bjgQ9Aoi4 iUMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912404; x=1704517204; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KfCrAxn+qVVomWZAVcoKxw5/zF1Ne58DSw43xSkBu64=; b=RDfrlcA+/1eQAKE/wQK+JCnE0Swi4R4+/g+TtMwQOZTr3g50ffeaR9qk5zYH8pn2ZG 4bOMbd8C9xUTPyxuNraO0VO2a5Q904f9Lkqn0SHN5d3hwRwYtYcTHDHhcAnEtjvYKrCe /xFe9LLj6prYM2+9zFbol2n8V6ZofUCGzfxWxp8GqXQac0sQsa9ZFVmbAEFXoXSL/vXs wVcal3kz3/5rBRhvSUghv4pUUvE7X2iUFWvw5LZcwJDGbr1HF2eEXjHBWFP/XXaprSYL +viH20AqAlp4Gc+jzRPS19cvplkzbUWPrZeQbDXV0tBvqQ1Py+cQwQu8h2seYWMDIR3u JZNQ== X-Gm-Message-State: AOJu0YxKYFer1N6SLLw9Hes3VB5P4ErRHrDD1zamWKDA1nIucyIMsEjA H1/+2fmiHUWsjGTU5gGr11v6ow2k44g= X-Google-Smtp-Source: AGHT+IEZvwHAXtuaXq7P0aEy0bbxBmpyCuwPRoX4Q5Dp4PmPO2i7QuReepftIRrd840q2nQ5crZbxg== X-Received: by 2002:a05:6a00:3984:b0:6da:5505:13c7 with SMTP id fi4-20020a056a00398400b006da550513c7mr1489146pfb.66.1703912403535; Fri, 29 Dec 2023 21:00:03 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.00.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:00:03 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 1/7] requirements.txt: bump all to latest Date: Fri, 29 Dec 2023 20:57:09 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:00:13 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62052 Pin mysqlclient as greater than 2.1.1 changes behavior Signed-off-by: Tim Orling --- requirements.txt | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/requirements.txt b/requirements.txt index 62b3f7b..20bcd30 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,8 +1,8 @@ -amqp==5.1.1 +amqp==5.2.0 asgiref==3.7.2 beautifulsoup4==4.12.2 -billiard==4.1.0 -celery==5.3.4 +billiard==4.2.0 +celery==5.3.6 click==8.1.7 click-didyoumean==0.3.0 click-plugins==1.1.1 @@ -10,29 +10,29 @@ click-repl==0.3.0 confusable-homoglyphs==3.2.0 diff-match-patch==20230430 Django>=4.2,<4.3 -django-appconf==1.0.5 -django-axes==6.1.0 -django-cors-headers==4.2.0 -django-ipware==5.0.0 +django-appconf==1.0.6 +django-axes==6.2.0 +django-cors-headers==4.3.1 +django-ipware==6.0.3 django-ranged-response==0.2.0 django-registration==3.4 -django-reversion==5.0.4 +django-reversion==5.0.9 django-reversion-compare==0.16.2 -django-simple-captcha==0.5.20 +django-simple-captcha==0.6.0 djangorestframework==3.14.0 -gitdb==4.0.10 -GitPython==3.1.37 -kombu==5.3.2 +gitdb==4.0.11 +GitPython==3.1.40 +kombu==5.3.4 mysqlclient==2.1.1 -Pillow==10.0.1 -prompt-toolkit==3.0.39 +Pillow==10.1.0 +prompt-toolkit==3.0.43 python-dateutil==2.8.2 -pytz==2023.3 +pytz==2023.3.post1 six==1.16.0 -smmap==5.0.0 -soupsieve==2.4.1 +smmap==5.0.1 +soupsieve==2.5 sqlparse==0.4.4 -typing_extensions==4.8.0 +typing_extensions==4.9.0 tzdata==2023.3 -vine==5.0.0 -wcwidth==0.2.8 +vine==5.1.0 +wcwidth==0.2.12 From patchwork Sat Dec 30 04:57:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A202C46CD2 for ; Sat, 30 Dec 2023 05:00:23 +0000 (UTC) Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by mx.groups.io with SMTP id smtpd.web10.179018.1703912421722678293 for ; Fri, 29 Dec 2023 21:00:21 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=ZDi4KoJ4; spf=pass (domain: gmail.com, ip: 209.85.210.173, mailfrom: ticotimo@gmail.com) Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-6d9a795cffbso3172695b3a.0 for ; Fri, 29 Dec 2023 21:00:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912421; x=1704517221; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=TELVvLV9C7tOcH3T4VaCQf40173Dd0YRYSar01PbQbY=; b=ZDi4KoJ4s5CocPbPPoeuLFDu66g9NcBFvcROJ+Po5ZVC9jLYgCF4q6IRSMJq10eTRW o/1obSdRupXEwApd/7+CGPkdwSXMW2bD9BtfRkc2X9arMa9INU1Czblbz2r7Kzg93C16 6hiOF3oRLsZLNc8MAjym2rtRiZzkyA2JcBjy/7UfMpOy1avFVN5OULuc4Mj+ryXObJMq bjrCmPQ/UEzf8OXk2MoV2qU+3hISth9TdXopxA5BWfBfdfZXTETS5yNzO2BQiNNqpTk8 +dRLVGTQPKHa5JGQ8G8Yl7XBd0IWxPucLYOT/Eh2q0hQ9rWPxajcn2hhPIszOzMfIn8A yDng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912421; x=1704517221; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TELVvLV9C7tOcH3T4VaCQf40173Dd0YRYSar01PbQbY=; b=AGwho2qltlN1xSIvbUaepbWJMBptsMDJMKPtfqDXF3/iqcPq48TcmkQczyvHviR/Ak M6Co2aeAUUoM1+yNUB3/a7rwLVqYTCZH3URE5bjpGQwViEbjNQiTJcJTrAWTVV/YT+cQ c2VJj6gdDSh+E1SoJzy6ZeJ3uN/JORPTQ9nke0G6ic+zWMxrET9hx07fkoUMZ89k9bWf OZySpGZvkD3wlyW9hfe1ZpnT6JqQRZBNUkixyPUgr20hDauIk90jO7iIUplK5KQz+f+V 5guPNVwjtcGNDSFS7xpeBPmXLE/46ttkUWCqTKpTc3Y6E6QuzLMWLzBqwPFIU9+/xKvX JveQ== X-Gm-Message-State: AOJu0Yzvgzd6nLfLzq816wNGTDcMpdhV83f0KUvsmHkFV77ru6bUzEQN r6c5Fq+2EtVZ7Ay3zfk8rP/Y5QMDQOE= X-Google-Smtp-Source: AGHT+IFwfT6gzTgps0WX+qJipk8n6U6a6ogEheTlNE7frN5laD+LLrSXFnmrd5YKQXrfn1VfIQUIlA== X-Received: by 2002:a05:6a00:1d81:b0:6d9:b42a:a6f with SMTP id z1-20020a056a001d8100b006d9b42a0a6fmr11430173pfw.29.1703912420710; Fri, 29 Dec 2023 21:00:20 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.00.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:00:20 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 2/7] layerindex/views.py: fix parse_view typo Date: Fri, 29 Dec 2023 20:57:11 -0800 Message-Id: <01094d0e2a2c4f8df9feea877b2159677c642ab8.1703911977.git.tim.orling@konsulko.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:00:23 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62053 The Branch Comparison view would throw an error because of a parser_view instead of parse_view typo. [YOCTO #15332] Signed-off-by: Tim Orling --- layerindex/views.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/layerindex/views.py b/layerindex/views.py index e660578..2e87181 100644 --- a/layerindex/views.py +++ b/layerindex/views.py @@ -1791,7 +1791,7 @@ class BranchCompareView(FormView): for _, recipe in sorted(recipes.items(), key=lambda item: item[0]): recipe.from_versions = sorted(recipe.from_versions, key=lambda item: parse_version(item.pv)) from_version_exprs = [x.version_expr() for x in recipe.from_versions] - recipe.to_versions = sorted(recipe.to_versions, key=lambda item: parser_version(item.pv)) + recipe.to_versions = sorted(recipe.to_versions, key=lambda item: parse_version(item.pv)) to_version_exprs = [x.version_expr() for x in recipe.to_versions] if not from_version_exprs: added.append(recipe) From patchwork Sat Dec 30 04:57:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26032C46CD2 for ; Sat, 30 Dec 2023 05:00:53 +0000 (UTC) Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by mx.groups.io with SMTP id smtpd.web11.179129.1703912450216474707 for ; Fri, 29 Dec 2023 21:00:50 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GYyJfybY; spf=pass (domain: gmail.com, ip: 209.85.210.173, mailfrom: ticotimo@gmail.com) Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-6d99980b2e0so4774014b3a.2 for ; Fri, 29 Dec 2023 21:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912449; x=1704517249; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=FgZoMkrOCxiY4QtNPEqasv7QeQtDHfVIoqE3B6ucRPM=; b=GYyJfybYbO14elob/xNNQYMJoCJM42HMt6TuaeZ8/cmh+LYf7xjmLmR27y+gnczuRG DeG69XDz91NqokgkIa76+Ps8CN/886egSa+PRBf0jUpDekyUE5bbaoQZ1Urn70etaVjV gxXDLXy/ZU4L6ZYAlaHtKgKEOXfmWwnqDIfEsfgGx2uxCfHHN3BAl43f6qkDpQ5iJHrD x6UNvnhKHYBZRKtDeJxa9bvFrhf0BVKaWF5mAd2VRvHgMX3SxDUeHCyD1KwtUvo/ew2G dhIcglYoubCSaiMm+jBocMXD8OxILX3RRC8hPOrXFUOLniTx/HU8b16O2LkeCHKnIEK+ Vg1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912449; x=1704517249; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FgZoMkrOCxiY4QtNPEqasv7QeQtDHfVIoqE3B6ucRPM=; b=RQnCjmmYCNl4PiQYqKD4FRSaHR4mSXYT5n7GYDn3CdMrDAiXw6iVEv1d7mgCJJGNXq 4SGUiyuNtV5bw1NdK3OhLivI0xJQLuGfSvFjt+256/lCmBUeof3PZowD6YGn8yrWLzp5 MJCSiBMiz6ZuwL5odUUCOusr3YrLwKpqVvoNA0gshSM+h7HULBC1SZ1hFFSmjMuMP4WW xvZWtO6lUrNwXZqvNrSdyCCtwl1y9BbmNiW3SmS6L1aM8ZSCrKq0SOcgm/GdE8ai7yVz BxbzDVQcQRzYSjKEXBUAAEX/ZOckb5QfhJJCZHwsGza9IGHlDBnOkPiLDt9U6/gJryH2 Nwdw== X-Gm-Message-State: AOJu0YxsaWegVwi8tPzNjO5Y6gdZ8zkrnK7fZQyP/5DZmQ/wDhsv1Fnu p46Cj+zgGl/n7eFvTrMgU9SOEyhjsao= X-Google-Smtp-Source: AGHT+IHUwKKcjOUzUJsFkOckdSqYBzJHP5qf4jSynyMc+eAb0CtnmMrcllX8yzs6kkV2noZmEiXq1Q== X-Received: by 2002:a05:6a00:1ca8:b0:6d9:af4e:804 with SMTP id y40-20020a056a001ca800b006d9af4e0804mr10362952pfw.35.1703912449289; Fri, 29 Dec 2023 21:00:49 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.00.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:00:48 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 3/7] docker/settings.py: fix RABBIT_BROKER TypeError Date: Fri, 29 Dec 2023 20:57:13 -0800 Message-Id: <00b024cff8768852bb0f57a3b863dc615c5b2b77.1703911977.git.tim.orling@konsulko.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:00:53 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62054 Within the layersapp container, if you try to run layerindex/update.py: File "/opt/layerindex/settings.py", line 280, in RABBIT_BROKER = 'amqp://' + os.getenv('RABBITMQ_DEFAULT_USER') + ':' + os.getenv('RABBITMQ_DEFAULT_PASS') + '@layersrabbit:5672/' TypeError: can only concatenate str (not "NoneType") to str Signed-off-by: Tim Orling --- docker/settings.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/settings.py b/docker/settings.py index 33ab332..c202585 100644 --- a/docker/settings.py +++ b/docker/settings.py @@ -276,7 +276,7 @@ SUBMIT_EMAIL_SUBJECT = 'OE Layerindex layer submission' SEND_PUBLISH_EMAIL = True # RabbitMQ settings -RABBIT_BROKER = 'amqp://' + os.getenv('RABBITMQ_DEFAULT_USER') + ':' + os.getenv('RABBITMQ_DEFAULT_PASS') + '@layersrabbit:5672/' +RABBIT_BROKER = 'amqp://' + os.getenv('RABBITMQ_DEFAULT_USER', 'guest') + ':' + os.getenv('RABBITMQ_DEFAULT_PASS', 'guest') + '@layersrabbit:5672/' RABBIT_BACKEND = 'rpc://layersrabbit/' # Used for fetching repo From patchwork Sat Dec 30 04:57:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D9ACC46CD2 for ; Sat, 30 Dec 2023 05:02:03 +0000 (UTC) Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by mx.groups.io with SMTP id smtpd.web10.179038.1703912522859845292 for ; Fri, 29 Dec 2023 21:02:02 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=Y0pomheN; spf=pass (domain: gmail.com, ip: 209.85.210.175, mailfrom: ticotimo@gmail.com) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6d9b37f4804so2768297b3a.1 for ; Fri, 29 Dec 2023 21:02:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912522; x=1704517322; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=R8DgZbxxG9wIzpzrde7fXuVaqT/AO7X3xhPQ8FQisyI=; b=Y0pomheN/N+SZKjgtoJ6BKtm8cLBs90yLxpS7cmCVNT0qh+3N09nxr1CVrhDZ1cXKY h0H5tIKC4zgR21OL+fbNW+14o7voamDeeylXMyqWVHNYFmXUgl+eaeB25G+wncjDExt/ YSTNlucP9rXhYMNOWfkPUaApVc1N4QVTNDPTFQP1qStYlyNPKrHS5skrKmJE1vEtHZYF zXhpPq6S4KPWDDJR612z5hhmOWL56DlXQ9bQF1o8JgWXbmZMPwPbGJJ8ZFOjBnfVFtaj r02fQR39rJUsVGPP0GwZI5fVkDuI+eQt0hhsH6dEwYhcpETUuVfWbNnVOzcqgnbf8mSb 0kuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912522; x=1704517322; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=R8DgZbxxG9wIzpzrde7fXuVaqT/AO7X3xhPQ8FQisyI=; b=D9E6Xk4s9sOe8+iJNWKbsGu2/d6GkwYDjUUDYqNP59fIm+O9jwO0PoW/cVpS/gtnSb i7GrwuNG68ovNga8GKG/GgAZR0skDliingXfJWSin4yvcXoVk8gYrZ1Jf8DMC0mwv3JK 9eyev/kj08I60DMbcNqVyQsch9/E1006ggg9KmbDhdJwR6dCxtSViPCQm06qj2rQKZEX /Pfr46IqscnEi66ZmNPVuVhFv6NEevjhwBHY1VywbBne2e/JfKdY7AMOWpmDb6tzp/ZF M48FkuQM4zeliakZeZeiIcshqsU9mjal54jAb1EtUArVBAKA3qX0oyDQamnar1u2YHOv SJBA== X-Gm-Message-State: AOJu0YxuIjzs8kwS9NCOoyxxupfyi5AGDzcWMbZtLEcYvQwyNk2hfKf3 K5ItWJq9hw9sH2K+fRdjLuuMX6T9iv8= X-Google-Smtp-Source: AGHT+IHih1XS3MhqW5pTmyWmIUqysgSZEM/FapfAzjDT+myxQTi5cOMJ77mk6IqkhOingPKB0ZZU5g== X-Received: by 2002:a05:6a20:748c:b0:196:444a:944 with SMTP id p12-20020a056a20748c00b00196444a0944mr5385445pzd.52.1703912521794; Fri, 29 Dec 2023 21:02:01 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.02.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:02:01 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 4/7] dockersetup.py: enable production Let's Encrypt Date: Fri, 29 Dec 2023 20:57:15 -0800 Message-Id: <203bbdb44fadb12cd7706601c510ac04e4c2fdfa.1703911977.git.tim.orling@konsulko.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:02:03 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62055 The '--staging' argument to certbot has now been changed to '--test-cert'. We previously only allowed using the dockersetup.py tool to create Staging environment certs, which are still marked as invalid by browsers. Add a '--letsencrypt-production' knob to allow for valid, trusted certs to be created. If they already exist in the workspace and have not expired, re-use them (to avoid hitting rate limits). Continue to '--force-renewal' for staging certs. NOTE: If you have previously created staging certs in your workspace, you will want to clean docker/certs before creating production certs for the same domain. Certbot will not overwrite those staging certs and the newly created ones will not be in the path passed in by dockersetup.py. Signed-off-by: Tim Orling --- dockersetup.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/dockersetup.py b/dockersetup.py index d6e8f33..56458d3 100755 --- a/dockersetup.py +++ b/dockersetup.py @@ -66,6 +66,7 @@ def get_args(): parser.add_argument('--cert', type=str, help='Existing SSL certificate to use for HTTPS web serving', required=False) parser.add_argument('--cert-key', type=str, help='Existing SSL certificate key to use for HTTPS web serving', required=False) parser.add_argument('--letsencrypt', action="store_true", default=False, help='Use Let\'s Encrypt for HTTPS') + parser.add_argument('--letsencrypt-production', action="store_true", default=False, help='Use Production server for Let\'s Encrypt. Default is %(default)s') parser.add_argument('--no-migrate', action="store_true", default=False, help='Skip running database migrations') parser.add_argument('--no-admin-user', action="store_true", default=False, help='Skip adding admin user') parser.add_argument('--no-connectivity', action="store_true", default=False, help='Skip checking external network connectivity') @@ -473,7 +474,7 @@ def edit_dockerfile_web(hostname, no_https): writefile("Dockerfile.web", ''.join(newlines)) -def setup_https(hostname, http_port, https_port, letsencrypt, cert, cert_key, emailaddr): +def setup_https(hostname, http_port, https_port, letsencrypt, letsencrypt_production, cert, cert_key, emailaddr): local_cert_dir = os.path.abspath('docker/certs') container_cert_dir = '/opt/cert' if letsencrypt: @@ -548,7 +549,7 @@ def setup_https(hostname, http_port, https_port, letsencrypt, cert, cert_key, em shutil.rmtree(tempdir) # Now run certbot to register SSL certificate - staging_arg = '--staging' + staging_arg = '--force-renewal --test-cert' if not letsencrypt_production else '--keep-until-expiring' if emailaddr: email_arg = '--email %s' % quote(emailaddr) else: @@ -560,7 +561,7 @@ def setup_https(hostname, http_port, https_port, letsencrypt, cert, cert_key, em -d %s \ --rsa-key-size 4096 \ --agree-tos \ - --force-renewal" layerscertbot' % (staging_arg, email_arg, quote(hostname)), shell=True) + " layerscertbot' % (staging_arg, email_arg, quote(hostname)), shell=True) if return_code != 0: print("Running certbot failed") sys.exit(1) @@ -757,7 +758,7 @@ else: edit_options_file(args.project_name) if not args.no_https: - setup_https(args.hostname, http_port, https_port, args.letsencrypt, args.cert, args.cert_key, emailaddr) + setup_https(args.hostname, http_port, https_port, args.letsencrypt, args.letsencrypt_production, args.cert, args.cert_key, emailaddr) ## Start up containers return_code = subprocess.call(['docker-compose', 'up', '-d', '--build'], shell=False) From patchwork Sat Dec 30 04:57:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36D6DC46CD2 for ; Sat, 30 Dec 2023 05:03:13 +0000 (UTC) Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by mx.groups.io with SMTP id smtpd.web10.179049.1703912591135641552 for ; Fri, 29 Dec 2023 21:03:11 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=VK3sHipQ; spf=pass (domain: gmail.com, ip: 209.85.215.176, mailfrom: ticotimo@gmail.com) Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-5ce0efd60ddso1528710a12.0 for ; Fri, 29 Dec 2023 21:03:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912590; x=1704517390; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=6PiepnzD4gDyD1DzcoJyt7ekSIk2GucB65VrpOxwtNU=; b=VK3sHipQs2qGhKLLdg+bRlxac50aF3XJU+MZ+BUxDDUEhG/w0a50C0qx0M+XnLp7nj 9cfhyXHakWqxIEDZWcIMoqGOcFaf8eGjHlY7dbZ4tJob82GcjwiOR3AZUA+/14N5SVnr 7HhE9HCTiMW+LqfYC7HwxNPBJ2bkGnvqfzKtGTXTOHMRcdmCwEFFuIMPwjX7iDBr0fRo BQ3RnUo3XtPmqh+eux5JkymQEqIGhS2hmof5eFmOD2MsBaDyT8Ha34C1jnL4pmjyVuz1 K2IepovRK8fV0/mGqFlyDjlLNe/J4vjWf17ARqS1XXzhvgdesYSy/5zr9y0w72O1XvZY fnEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912590; x=1704517390; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6PiepnzD4gDyD1DzcoJyt7ekSIk2GucB65VrpOxwtNU=; b=Dr96aHL/UomwG5hbFL3w8te3oSSJLlJat/qV6NVo/yn7p2xrHh7RbSfAaEWbaHllE5 pT0R2/UOtETk+0K8J8x1Krggp/T28dLU8SBGAljv0gztSR6+STHyhmB12eIvw9HNHHAi CImOt9qI1p8+wDgjVT4a4AUpzMYhar5hoAIs7Zp8bFU0FtKjkjjh8Wouo9M8MjaznEoH EgDAg8/izPEy8ozZfipDBfHjHM/nb4od2++S0EOudvywHORw3/hyy/rBfqgLI1HmLjYw 2M3muRMC/JIvVYVi7LGoYRdewOBv6RzzsXTcVIV/w32NoZnOsyjWLr8XK9p0hgkMsnkv GsfQ== X-Gm-Message-State: AOJu0YwIFJwKyMai17NPAbEs6/0TeLTT9UkPkN9UsIMQ5wmbriCgjybo uJSOqmOTIN0jOK4yWjZ9r16DLIMNWr8= X-Google-Smtp-Source: AGHT+IFmLbkzj0/zgWuwA4iRVIzZyqvAQNf1xcp1vcxoNYkIAKOF0DYoEkE/39fF19jIQJAPVFs3Sw== X-Received: by 2002:a05:6a21:819f:b0:196:a4c2:8bc4 with SMTP id pd31-20020a056a21819f00b00196a4c28bc4mr263000pzb.118.1703912590047; Fri, 29 Dec 2023 21:03:10 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.03.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:03:09 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 5/7] layerindex: Add actual_branch to forms and views Date: Fri, 29 Dec 2023 20:57:17 -0800 Message-Id: <083c23476450ea85e7d3f4a4aff4bb619c7b59fb.1703911977.git.tim.orling@konsulko.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:03:13 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62056 For layers which do not follow standard branch names (including the inclusive naming move away from "master" to "main") we have the actual_branch field set in a LayerBranch object. Previously this was only exposed via the admin interface. Allow layer maintainers (including upon submitting a new layer) to set the 'Actual branch' in the web UI. Add a check to make sure the actual_branch is a valid branch name using 'git check-ref-format --branch ' since we are not using full refs. [YOCTO #8008] NOTE: Only existing LayerBranches will be editable. A new layer can be submitted with a different branch for "master", but only the "master" LayerBranch will be created. Further changes to the update.py script will be needed to make creation of new stable branches with an actual_branch possible. Signed-off-by: Tim Orling --- layerindex/forms.py | 15 +++++++++++++++ layerindex/views.py | 5 +++++ 2 files changed, 20 insertions(+) diff --git a/layerindex/forms.py b/layerindex/forms.py index 9615859..3b764f1 100644 --- a/layerindex/forms.py +++ b/layerindex/forms.py @@ -76,6 +76,7 @@ LayerMaintainerFormSet = inlineformset_factory(LayerBranch, LayerMaintainer, for class EditLayerForm(StyledModelForm): # Additional form fields vcs_subdir = forms.CharField(label='Repository subdirectory', max_length=40, required=False, help_text='Subdirectory within the repository where the layer is located, if not in the root (usually only used if the repository contains more than one layer)') + actual_branch = forms.CharField(label='Actual branch', max_length=80, required=False, help_text='Name of the actual branch in the repository matching the core branch (e.g. the development branch is "master" by default)') deps = forms.ModelMultipleChoiceField(label='Other layers this layer depends upon', queryset=LayerItem.objects.filter(comparison=False), required=False) captcha = CaptchaField(label='Verification', help_text='Please enter the letters displayed for verification purposes', error_messages={'invalid':'Incorrect entry, please try again'}) @@ -98,11 +99,16 @@ class EditLayerForm(StyledModelForm): field_order.pop(field_order.index('vcs_subdir')) name_pos = field_order.index('vcs_url') + 1 field_order.insert(name_pos, 'vcs_subdir') + # Ensure actual branch appears after repo subdir + field_order.pop(field_order.index('actual_branch')) + name_pos = name_pos + 1 + field_order.insert(name_pos, 'actual_branch') new_fields = OrderedDict() for field in field_order: new_fields[field] = self.fields[field] self.fields = new_fields self.fields['vcs_subdir'].initial = layerbranch.vcs_subdir + self.fields['actual_branch'].initial = layerbranch.actual_branch self.was_saved = False self.allow_base_type = allow_base_type @@ -178,6 +184,15 @@ class EditLayerForm(StyledModelForm): val(usage) return usage + def clean_actual_branch(self): + import subprocess + actual_branch = self.cleaned_data['actual_branch'].strip() + process = subprocess.Popen(["git", "check-ref-format", "--branch", actual_branch]) + exit_status = process.wait() + if exit_status != 0: + raise forms.ValidationError("Actual branch should be a valid git branch short name") + return actual_branch + class EditNoteForm(StyledModelForm): class Meta: diff --git a/layerindex/views.py b/layerindex/views.py index 2e87181..1a29dde 100644 --- a/layerindex/views.py +++ b/layerindex/views.py @@ -162,6 +162,11 @@ def edit_layer_view(request, template_name, branch='master', slug=None): layerbranch.vcs_subdir = new_subdir reset_last_rev = True layerbranch.save() + new_actual_branch = form.cleaned_data['actual_branch'] + if layerbranch.actual_branch != new_actual_branch: + layerbranch.actual_branch = new_actual_branch + reset_last_rev = True + layerbranch.save() maintainerformset.save() if slug: new_deps = form.cleaned_data['deps'] From patchwork Sat Dec 30 04:57:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37110 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A830C46CD2 for ; Sat, 30 Dec 2023 05:04:53 +0000 (UTC) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by mx.groups.io with SMTP id smtpd.web11.179178.1703912683634897657 for ; Fri, 29 Dec 2023 21:04:43 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=GqMbaftw; spf=pass (domain: gmail.com, ip: 209.85.167.174, mailfrom: ticotimo@gmail.com) Received: by mail-oi1-f174.google.com with SMTP id 5614622812f47-3bbbc6b4ed1so2845687b6e.2 for ; Fri, 29 Dec 2023 21:04:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912682; x=1704517482; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=FL+4Ox9+d59P/2wPUcgMs3UsmQ8g2AgUvOpIscot+hk=; b=GqMbaftwaIek9sVH8MS4ZwETUIJbJ5Jj16uFGPoiQxqV+FxV7ownNGKy5jEsZuFlE1 oEj4Jo1ejHJ/5MMkDYxFu4FzPRF9zQyQCGkF1GbwRV0cA8EO+oaF4yK1Zci3vAkw4x4e L3WESd4NltY8CHurIHzHkrjw3IlC6eblcxyhjvClHLvZbET8TR9vipNSq/ihk4SYbd9P w9a65N00EacZOuDPgSMb+qT9U/D9xTZiEllTUYmujWgtMkhiMxgJppX9FWZW1ScjsKiq tXHkczpDRtZX6C9VDvXDWxksedSmPIaoMB//JBvRDJwdbctFgDEABFz9L9UcP8TWQe2I WCCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912682; x=1704517482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FL+4Ox9+d59P/2wPUcgMs3UsmQ8g2AgUvOpIscot+hk=; b=vkDPUz0+vxyrm/UeYivcA80+5KaFArRegjQCCVw2fCna4LEC81Wh3QoLyi2abrb+fF VfdIqwqAN3jtJAhgD5dOtyFiarJaSQ4o49OiuEfQLbc5AveXVUBP4JjL20ZuR3namxy0 vRlrceZ0rnzNF3s7bnQTG1NPWzUqbTc3UcBdFjnCc5nMr4EUfBbf9+BB9XI8j0s+VwU/ 5EKM8dqaT6TLhp/DqVh18ctf4MGHAF0vqrOErzXb7J9F/daLlXe2xGBknlJiu53J24OO Q2kbNR0PTz/q9YGkcInEQIsqzy84u8M8W+38C4g2zdVkcm4cMDZhFtVlPPdgKEoUemoR uoDA== X-Gm-Message-State: AOJu0YxLhyZkQ9N8G9QqZ1HH7Ubl3kQ4Cfc2XdI+injgS3BNK5Rgt4mP r/+ybwNeHdzeUHM86eMjRc0i4dZYDnA= X-Google-Smtp-Source: AGHT+IH+zLPb0/Ze+37zCi5Aqqj4x4szBL2jop/Ut2WMKUAb1Opj8ByY1N+FcRjxbnrJdlAiWvGWZg== X-Received: by 2002:a05:6358:9106:b0:174:fdd8:7cae with SMTP id q6-20020a056358910600b00174fdd87caemr5253448rwq.36.1703912682238; Fri, 29 Dec 2023 21:04:42 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.04.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:04:41 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 6/7] layerindex: add Update Layer UI feature Date: Fri, 29 Dec 2023 20:57:19 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:04:53 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62057 * Add an "Update Layer" button to the layer detail view. - This allows a user that is a member of is_staff to trigger an update of the current layer (for the current branch) * Add an "Update Layer" button to the reviewdetail view - This allows a user that is a member of is_staff and has publish_layer permissions to trigger an update attempt of the layer under review (even in the un-published state) * The update is run as a task with Celery NOTE: You must have the RABBITMQ_ and DATABASE_ credentials set correctly in the docker/settings.py file or set via environment variables or you will get authentication errors talking to layersdb or layersrabbit containers. [YOCTO #12484] layerindex/views.py: add update_layer_view layerindex/urls.py: add update_layer_view layerindex/urls_branch.py: add update_layer_view templates/layerindex/reviewdetail.html: add Update Layer button templates/layerindex/detail.html: add Update Layer button templates/layerindex: add updatelayer.html TODO: While the update is happening, the AJAX rendering of the update.log is showing the b'' characters and not adding any new lines. If you go back to the same task view afterwards, the log is rendered as expected. TODO: After the update is completed, it would be nice to have a button to return you to the page from where you called the "Update Layer". Signed-off-by: Tim Orling --- layerindex/urls.py | 2 +- layerindex/urls_branch.py | 3 +- layerindex/views.py | 35 ++++++++ templates/layerindex/detail.html | 3 + templates/layerindex/reviewdetail.html | 3 + templates/layerindex/updatelayer.html | 119 +++++++++++++++++++++++++ 6 files changed, 163 insertions(+), 2 deletions(-) create mode 100644 templates/layerindex/updatelayer.html diff --git a/layerindex/urls.py b/layerindex/urls.py index cee3d2a..1fcd11d 100644 --- a/layerindex/urls.py +++ b/layerindex/urls.py @@ -16,7 +16,7 @@ from layerindex.views import LayerListView, LayerReviewListView, LayerReviewDeta ClassicRecipeSearchView, ClassicRecipeDetailView, ClassicRecipeStatsView, LayerUpdateDetailView, UpdateListView, \ UpdateDetailView, StatsView, publish_view, LayerCheckListView, BBClassCheckListView, TaskStatusView, \ ComparisonRecipeSelectView, ComparisonRecipeSelectDetailView, task_log_view, task_stop_view, email_test_view, \ - BranchCompareView, RecipeDependenciesView + BranchCompareView, RecipeDependenciesView, update_layer_view from layerindex.models import LayerItem, Recipe, RecipeChangeset from rest_framework import routers from . import restviews diff --git a/layerindex/urls_branch.py b/layerindex/urls_branch.py index 6736f32..40cd915 100644 --- a/layerindex/urls_branch.py +++ b/layerindex/urls_branch.py @@ -8,7 +8,7 @@ from django.views.defaults import page_not_found from django.urls import include, re_path, reverse_lazy -from layerindex.views import LayerListView, RecipeSearchView, MachineSearchView, DistroSearchView, ClassSearchView, LayerDetailView, edit_layer_view, delete_layer_view, edit_layernote_view, delete_layernote_view, RedirectParamsView, DuplicatesView, LayerUpdateDetailView, layer_export_recipes_csv_view, comparison_update_view +from layerindex.views import LayerListView, RecipeSearchView, MachineSearchView, DistroSearchView, ClassSearchView, LayerDetailView, edit_layer_view, delete_layer_view, edit_layernote_view, delete_layernote_view, RedirectParamsView, DuplicatesView, LayerUpdateDetailView, layer_export_recipes_csv_view, comparison_update_view, update_layer_view urlpatterns = [ re_path(r'^$', @@ -41,6 +41,7 @@ urlpatterns = [ template_name='layerindex/classes.html'), name='class_search'), re_path(r'^edit/(?P[-\w]+)/$', edit_layer_view, {'template_name': 'layerindex/editlayer.html'}, name="edit_layer"), + re_path(r'^update/(?P[-\w]+)/$', update_layer_view, {'template_name': 'layerindex/updatelayer.html'}, name="update_layer"), re_path(r'^duplicates/$', DuplicatesView.as_view( template_name='layerindex/duplicates.html'), diff --git a/layerindex/views.py b/layerindex/views.py index 1a29dde..25c045e 100644 --- a/layerindex/views.py +++ b/layerindex/views.py @@ -121,6 +121,41 @@ def delete_layer_view(request, template_name, slug): 'cancel_url': layeritem.get_absolute_url() }) +def update_layer_view(request, template_name, branch='master', slug=None): + if not (request.user.is_authenticated and request.user.is_staff): + raise PermissionDenied + return_url = None + branchobj = Branch.objects.filter(name=branch)[:1].get() + if slug: + # Existing Layer Update + layeritem = get_object_or_404(LayerItem, name=slug) + layerbranch = get_object_or_404(LayerBranch, layer=layeritem, branch=branchobj) + returnto = request.GET.get('returnto', 'layer_item') + if returnto: + if returnto == 'layer_review': + return_url = reverse_lazy(returnto, args=(layeritem.name,)) + else: + return_url = reverse_lazy(returnto, args=(branch, layeritem.name)) + else: + # Pre-Publish Layer Update attempt + layeritem = LayerItem() + layerbranch = LayerBranch(layer=layeritem, branch=branchobj) + + from celery import uuid + + cmd = 'layerindex/update.py --layer %s --branch %s;' % (layeritem.name, branch) + + task_id = uuid() + # Create this here first, because inside the task we don't have all of the required info + update = Update(task_id=task_id) + update.started = datetime.now() + update.triggered_by = request.user + update.save() + + res = tasks.run_update_command.apply_async((branch, cmd), task_id=task_id) + + return HttpResponseRedirect(reverse_lazy('task_status', kwargs={'task_id': task_id})) + def edit_layer_view(request, template_name, branch='master', slug=None): return_url = None branchobj = Branch.objects.filter(name=branch)[:1].get() diff --git a/templates/layerindex/detail.html b/templates/layerindex/detail.html index 12ce156..0fa8caf 100644 --- a/templates/layerindex/detail.html +++ b/templates/layerindex/detail.html @@ -38,6 +38,9 @@ {% endif %} {% if user.is_authenticated %} + {% if user.is_staff %} + Update layer + {% endif %} {% if perms.layerindex.publish_layer or useredit %} Edit layer {% if layeritem.layernote_set.count == 0 %} diff --git a/templates/layerindex/reviewdetail.html b/templates/layerindex/reviewdetail.html index 8baab7c..39501be 100644 --- a/templates/layerindex/reviewdetail.html +++ b/templates/layerindex/reviewdetail.html @@ -47,6 +47,9 @@ Delete layer Publish layer {% endif %} + {% if user.is_staff %} + Update layer + {% endif %} {% endif %} diff --git a/templates/layerindex/updatelayer.html b/templates/layerindex/updatelayer.html new file mode 100644 index 0000000..799fb51 --- /dev/null +++ b/templates/layerindex/updatelayer.html @@ -0,0 +1,119 @@ +{% extends "base.html" %} +{% load i18n %} +{% load extrafilters %} + +{% comment %} + + layerindex-web - update page + + Copyright (C) 2016, 2018 Intel Corporation + Licensed under the MIT license, see COPYING.MIT for details + +{% endcomment %} + + + +{% block content %} +{% autoescape on %} + +
    +
  • Updates
    • +
  • {{ update.started }}
    • +
+ + +

{{ update.started }} {% if update.reload %}(reload){% endif %} +{% if update.finished %}{% if update.retcode < 0 %}TERMINATED ({{ update.retcode }}){% elif update.retcode %}FAILED{% endif %}{% endif %} +

+ +{% if update.log %} + 
{{ update.log }}
+{% endif %} + +{% for layerupdate in layerupdates %} + + + {% with layerbranch_exists=layerupdate.layerbranch_exists %} + 0 or layerupdate.retcode != 0 %} class="error"{% elif layerupdate.warnings %} class="warning"{% endif %}> + {% if layerbranch_exists %}{% endif %}{{ layerupdate.layer.name }} {{ layerupdate.branch.name }}{% if layerbranch_exists %}{% endif %} + {% if layerupdate.vcs_before_rev != layerupdate.vcs_after_rev %} + + {% if layerbranch_exists %} + {% with before_url=layerupdate.vcs_before_commit_url after_url=layerupdate.vcs_after_commit_url %} + {% if before_url %}{% endif %}{{ layerupdate.vcs_before_rev|truncatesimple:10 }}{% if before_url %}{% endif %} → {% if after_url %}{% endif %}{{ layerupdate.vcs_after_rev|truncatesimple:10 }}{% if after_url %}{% endif %} + {% endwith %} + {% else %} + {{ layerupdate.vcs_before_rev|truncatesimple:10 }} → {{ layerupdate.vcs_after_rev|truncatesimple:10 }} + {% endif %} + + {% endif %} + + {% endwith %} + + + {% if layerupdate.log %} + + {% endif %} + +
+ 
{{ layerupdate.log }}
+
+{% endfor %} + +{% if not update.log and not layerupdates %} +

No messages or layer updates

+{% endif %} + +{% if update.comparisonrecipeupdate_set.exists %} +

Updated comparison recipes

+
    +{% for recipeupdate in update.comparisonrecipeupdate_set.all %} +
  • {{ recipeupdate.recipe.pn }} {% if recipeupdate.meta_updated and recipeupdate.link_updated %}(meta, link){% elif recipeupdate.link_updated %}(link){% elif recipeupdate.meta_updated %}(meta){% endif %}
    • +{% endfor %} +
+{% endif %} + +{% endautoescape %} + +{% endblock %} +{% if updates.count > 0 %} +
+ + + + + + + + + + + + {% for update in updates %} + + + + + + {% endfor %} + +
Date/timeErrorsWarnings
+ {% if update.log %} + {{ update.started }}{% if update.update.reload %} (reload){% endif%} + {% else %} + {{ update.started }}{% if update.update.reload %} (reload){% endif%} + {% endif %} + {% if update.errors %}{{ update.errors }}{% endif %}{% if update.warnings %}{{ update.warnings }}{% endif %}
+
+{% endif %} + +{% block scripts %} +{% endblock %} From patchwork Sat Dec 30 04:57:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Orling X-Patchwork-Id: 37111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35126C46CD2 for ; Sat, 30 Dec 2023 05:06:53 +0000 (UTC) Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) by mx.groups.io with SMTP id smtpd.web10.179090.1703912805630791552 for ; Fri, 29 Dec 2023 21:06:45 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20230601 header.b=fpBKwq3k; spf=pass (domain: gmail.com, ip: 209.85.167.182, mailfrom: ticotimo@gmail.com) Received: by mail-oi1-f182.google.com with SMTP id 5614622812f47-3bb9d54575cso4363561b6e.2 for ; Fri, 29 Dec 2023 21:06:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703912804; x=1704517604; darn=lists.yoctoproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=8GMUfkHv/jHlSRlKb0PxsER6Pemf6fEvvd2a6KhA3aQ=; b=fpBKwq3kssPoWUOR37uK/XcAmtVzNOaerWrQobGZ1xRFeANrxzjHkAvsilsmC3AEgy neZdNb/i9WDmkjE/Ykz+YS95XjQjFhbEs9eZR6wPOeoSXrh57WTVGS+JWc7JsqoMEQ2Z 8F8PyS2+zsCwSSCF1N7MXE2uO4dv5DXmxupU+QxSrJdvJNe7VueveVErcsqx46h+8QXs 7/XLaKwZTtPcg4GfPYWdnZC6XS2AgVGD7pa0j2JNcZQPSMN1/rbzvuPtvQ+7n00SlG+O ZJ5P2Izs1Wg8j5g+5HsUHBd5XZ2BJyv6FzodIaqFOIqj2ZEZAS7Lf0csjXsifBCuEazB N76w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703912804; x=1704517604; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8GMUfkHv/jHlSRlKb0PxsER6Pemf6fEvvd2a6KhA3aQ=; b=ln+D46A2JF10ZeURN3Ex3uI+zMY0A+PCaWwgm14ePT8OnDBJrE9LRJaAoY4fTsaYqu thQfn/t3o9JfU8h7THQ4A2NbPY8fppWb/c8GZDqsKjalnc59F5e054ceAxBhwZBSKDrw cZc9ABYRDJBTT3+eyLL7ahFNCdVy6AW9RzBUC7mt1WRraZrAl3ZKIokfHo0Z6wpnoCgh olfPkDwKtxsRUkzcN00R71TPfvSKGgbo/g/WyYgT7M1oJoiuknkp6Ai14FwDowXkmiak uJ+4oppKe/a+6Ju4fwE/xf5ZL430aZd6ACSGBrJw/WID8BtYw8zEfBkcVNnLnE7Ck1gG cXUQ== X-Gm-Message-State: AOJu0Yy8AK9XugOU+twXu7NsF7xLO2asnxNPN37rDqrV+z3T/JSed2Ed vpSBNxrDaikoqWN0DaqVKMdj/F6t7zQ= X-Google-Smtp-Source: AGHT+IFpnnZ6P9K6+3+ShFnkyG/8SVe309hqW+I36qRVvBCqfPFVnUtYrEuEU0RF4SF14uuhShvppQ== X-Received: by 2002:a05:6358:e98:b0:172:cf9c:882a with SMTP id 24-20020a0563580e9800b00172cf9c882amr10025726rwg.32.1703912804469; Fri, 29 Dec 2023 21:06:44 -0800 (PST) Received: from chiron.hsd1.or.comcast.net ([2601:1c0:ca00:cea0:56fb:3e3a:cb3d:4a01]) by smtp.gmail.com with ESMTPSA id s19-20020aa78293000000b006d9a9727a8esm12124536pfm.178.2023.12.29.21.06.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Dec 2023 21:06:44 -0800 (PST) From: Tim Orling X-Google-Original-From: Tim Orling To: yocto@lists.yoctoproject.org Subject: [layerindex-web][PATCH 7/7] layerindex: improve updates for actual_branch Date: Fri, 29 Dec 2023 20:57:21 -0800 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 30 Dec 2023 05:06:53 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/yocto/message/62058 Some layers now have one branch with many supported LAYERSERIES_COMPAT. If this branch name does not match one of the stable releases, LayerBranches might not have been created. When actual_branch is set, it is only set in a LayerBranch object. We previously could not update (create) a stable branch with actual_branch except manually in the admin interface. Add --force-create option to be used in conjunction with --actual-branch (which already requires --branch) in the update.py script. This tells the script to ignore the fact that no layerbranch exists already. Add --actual-branch to update_layer.py so that we can create (and more importantly checkout) an actual_branch for the given stable --branch. Update utils.py to allow checking out of actual_branch when a LayerBranch does not yet exist. While we are at it, ensure that any Branch that is marked as no update will be skipped even with --force-create. The main reason that a Branch has updates disabled is because the bitbake or python syntax has changed enough to cause exceptions. This script can now be run with: ./layerindex/update.py \ --layer meta-weird-one \ --branch kirkstone \ --actual-branch=nonstandard \ --force-create Which will attempt to create a meta-weird-one:kirkstone layerbranch checked out at the 'nonstandard' branch from that layer's git repo. This allows layerindex admins to at least populate the database without tedious creation of layerbranches in the admin interface. Helps make the "branch mapping" actually work and be useful: [YOCTO #8008] Signed-off-by: Tim Orling --- layerindex/update.py | 39 ++++++++++++++++++++++++++++---------- layerindex/update_layer.py | 8 ++++++++ layerindex/utils.py | 7 +++++-- 3 files changed, 42 insertions(+), 12 deletions(-) diff --git a/layerindex/update.py b/layerindex/update.py index f9a216d..c1886bd 100755 --- a/layerindex/update.py +++ b/layerindex/update.py @@ -43,6 +43,8 @@ def prepare_update_layer_command(options, branch, layer, initial=False): else: cmdprefix = 'python3' cmd = '%s update_layer.py -l %s -b %s' % (cmdprefix, layer.name, branch.name) + if options.actual_branch and options.force_create: + cmd += ' --actual-branch=%s' % options.actual_branch if options.reload: cmd += ' --reload' if options.fullreload: @@ -86,15 +88,18 @@ def update_actual_branch(layerquery, fetchdir, branch, options, update_bitbake, logger.info("Skipping update actual_branch for %s - branch %s doesn't exist" % (layer.name, actual_branch)) continue layerbranch = layer.get_layerbranch(branch) - if not layerbranch: - logger.info("Skipping update actual_branch for %s - layerbranch %s doesn't exist" % (layer.name, branch)) - continue - if actual_branch != layerbranch.actual_branch: - logger.info("%s: %s.actual_branch: %s -> %s" % (layer.name, branch, layerbranch.actual_branch, actual_branch)) - layerbranch.actual_branch = actual_branch - to_save.add(layerbranch) + if not options.force_create: + if not layerbranch: + logger.info("Skipping update actual_branch for %s - layerbranch %s doesn't exist" % (layer.name, branch)) + continue + if actual_branch != layerbranch.actual_branch: + logger.info("%s: %s.actual_branch: %s -> %s" % (layer.name, branch, layerbranch.actual_branch, actual_branch)) + layerbranch.actual_branch = actual_branch + to_save.add(layerbranch) + else: + logger.info("%s: %s.actual_branch is already %s, so no change" % (layer.name, branch, actual_branch)) else: - logger.info("%s: %s.actual_branch is already %s, so no change" % (layer.name, branch, actual_branch)) + logger.info("%s: Allowing branch %s with actual_branch %s to attempt to be created" % (layer.name, branch, actual_branch)) # At last, do the save if not options.dryrun: @@ -169,6 +174,9 @@ def main(): parser.add_option("-a", "--actual-branch", help = "Update actual branch for layer and bitbake", action="store", dest="actual_branch", default='') + parser.add_option("", "--force-create", + help = "Create layer branch if it does not already exist", + action="store_true", dest="force_create", default=False) parser.add_option("-d", "--debug", help = "Enable debug output", action="store_const", const=logging.DEBUG, dest="loglevel", default=logging.INFO) @@ -197,6 +205,10 @@ def main(): if not utils.get_branch(branch): logger.error("Specified branch %s is not valid" % branch) sys.exit(1) + branchquery = Branch.objects.filter(updates_enabled=True).filter(name=branch) + if not branchquery.count() > 0: + logger.warning("Updates are disabled for specified branch %s" % branch) + sys.exit(1) else: branchquery = Branch.objects.filter(updates_enabled=True) branches = [branch.name for branch in branchquery] @@ -315,7 +327,7 @@ def main(): logger.error("No repositories could be fetched, exiting") sys.exit(1) - if options.actual_branch: + if options.actual_branch and not options.force_create: update_actual_branch(layerquery, fetchdir, branches[0], options, update_bitbake, bitbakepath) return @@ -389,6 +401,10 @@ def main(): logger.error('Repository %s is bare, not supported' % repodir) continue try: + # Allow stable branches to be created if actual_branch exists + if options.actual_branch: + branchname = options.actual_branch + branchdesc = "%s (%s)" % (branch, branchname) # Always get origin/branchname, so it raises error when branch doesn't exist when nocheckout topcommit = repo.commit('origin/%s' % branchname) if options.nocheckout: @@ -419,7 +435,10 @@ def main(): else: # Check out appropriate branch if not options.nocheckout: - utils.checkout_layer_branch(layerbranch, repodir, logger=logger) + if not options.actual_branch: + utils.checkout_layer_branch(layerbranch, repodir, logger=logger) + else: + utils.checkout_layer_branch(layerbranch, repodir, actual_branch=options.actual_branch, logger=logger) layerdir = os.path.join(repodir, layerbranch.vcs_subdir) if layerbranch.vcs_subdir and not os.path.exists(layerdir): print_subdir_error(newbranch, layer.name, layerbranch.vcs_subdir, branchdesc) diff --git a/layerindex/update_layer.py b/layerindex/update_layer.py index 6d73fad..4a67a50 100644 --- a/layerindex/update_layer.py +++ b/layerindex/update_layer.py @@ -272,6 +272,9 @@ def main(): parser.add_option("-i", "--initial", help = "Print initial values parsed from layer.conf only", action="store_true") + parser.add_option("-a", "--actual-branch", + help = "Specify actual_branch for git checkout", + action="store", dest="actual_branch", default=None) parser.add_option("-d", "--debug", help = "Enable debug output", action="store_const", const=logging.DEBUG, dest="loglevel", default=logging.INFO) @@ -326,6 +329,9 @@ def main(): if layerbranch.actual_branch: branchname = layerbranch.actual_branch branchdesc = "%s (%s)" % (options.branch, branchname) + elif options.actual_branch: + branchname = options.actual_branch + branchdesc = "%s (%s)" % (options.branch, branchname) # Collect repo info repo = git.Repo(repodir) @@ -347,6 +353,8 @@ def main(): layerbranch = LayerBranch() layerbranch.layer = layer layerbranch.branch = branch + if options.actual_branch: + layerbranch.actual_branch = options.actual_branch layerbranch_source = layer.get_layerbranch(branch) if not layerbranch_source: layerbranch_source = layer.get_layerbranch(None) diff --git a/layerindex/utils.py b/layerindex/utils.py index 0ea8e48..8bbd621 100644 --- a/layerindex/utils.py +++ b/layerindex/utils.py @@ -297,8 +297,11 @@ def checkout_repo(repodir, commit, logger, force=False): # Now check out the revision runcmd(['git', 'checkout', commit], repodir, logger=logger) -def checkout_layer_branch(layerbranch, repodir, logger=None): - branchname = layerbranch.get_checkout_branch() +def checkout_layer_branch(layerbranch, repodir, actual_branch=None, logger=None): + if actual_branch: + branchname = actual_branch + else: + branchname = layerbranch.get_checkout_branch() checkout_repo(repodir, 'origin/%s' % branchname, logger) def is_layer_valid(layerdir):