From patchwork Wed Aug 16 20:08:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 427E4C41513 for ; Wed, 16 Aug 2023 20:09:51 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (NAM04-BN8-obe.outbound.protection.outlook.com [40.107.100.134]) by mx.groups.io with SMTP id smtpd.web11.170823.1692216588445881618 for ; Wed, 16 Aug 2023 13:09:48 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=dhnJMtOd; spf=pass (domain: ni.com, ip: 40.107.100.134, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WwjiXOZrq/M2IhhgrdI71AEYCLO8N36ABkVm2UlQMwvxmVkdEKmsxksiAR7La417XTVWzY6C2AvB0Owdl/Jo7BSifhhSKk0wuICB2fbFmP4z7n2kYoCNfKUfwaksG9UkyxThDQugJwkeh/fefta5s8mw4jyI59tc8jFCbwWK5iOlO4M2x3E/IF/QjMBjefzt4Ai26f53jpl3q/h7QTqNIBEq6EAG93oXLEFQhF0D7M03pW3CLge4/wELDhTZAs1B5xtumjqrOLv2SA+XN/HrRcU9h/tQB423W93NqQRRkGImLW5JN5OU0TEUHesd/JjDc53HTzKtcasvZHemZ08UTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hc2R/mowLPwddwqosdWZmOLOzGBmS78DFJHZo51HWss=; b=CB1ywTr9fLK9pW1MU3Yu9udXaLDfJV3YL3Ou0FxsiURMk1EtEovyq8EipNXXH/BQwEGJ17cIMmc/PD/D1afOGQ07NBAFgFSzncLZiA5RB/XfXADu1SwzcAsL6smrznkNfuL7Pf4PH5hWDvHlM77XN/64c647kqEe8NrqAI0iXrmjlpU2pPN0e6V82N063u/oqGLHeONJCCARjVlOIAwOKCIFVP+ZBHhjBgW/HGVkwINFCV+U1kgk1iyAQk//VIb9baVRsrWr2JBG8e9Wr+LmLoTztsmPubwBf27Y1cdrLdhx3KgklDx4yN7frdwHeBaz17qgc4J3xk/MTjXITN6N+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hc2R/mowLPwddwqosdWZmOLOzGBmS78DFJHZo51HWss=; b=dhnJMtOdKJKB7UnLX23kkwZodr4atZyY4o7euMx5rtp8TRmBt8zSi3auTKnx0x/1Nl1Paoi0hYx69TlJ+NHds0+IpJ268veyNO1B4Gd6t1EOf8yHQBgf//WVan964aL+0mlXlkINiCCKa5vJtutva9eqbF2zy11gOOlHjDADEV0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by MN2PR04MB6400.namprd04.prod.outlook.com (2603:10b6:208:1a8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.26; Wed, 16 Aug 2023 20:09:46 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:09:46 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 1/7] bitbake.conf: Add new DEPLOY_DIR_FEED variables. Date: Wed, 16 Aug 2023 15:08:31 -0500 Message-ID: <20230816200931.2577181-2-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: SA1PR05CA0003.namprd05.prod.outlook.com (2603:10b6:806:2d2::14) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|MN2PR04MB6400:EE_ X-MS-Office365-Filtering-Correlation-Id: 9e48ae1d-f4c7-4e2e-5a02-08db9e94bcaf x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LcOhmZlUniAa4lHCwwdVOJDaBPZ15rJ+1K3nw14dwcTPk7FdhMlt0Ts4DmQOXq8GwICAxy3Rt85eLBKXDigeABbggb++B2tmxE7gxPWcyrKIhvYzD1ImZr9m6qnGof26CZr28NKRoQ6u40U4O+i5465En4EStKXq4Xv9MOGKqPbwYM+bUx0dAgkHHxUpTg5is9JHgtDXo1rrRjW6GDbU7CLTXUQl5RuGL/o4Y6gRVlsqrOqfqu7liAONgHSys/xrlbYkF1x48lrIMJv6Zdu01dNRIt9LUx+V5YdtRo66EjBWz0Ed3TSIBluJi0rJaQPksHF1bQ217wgMzM7HUNCfTqa2uLY0txn888kOaYm1NgXbhzh/Ir6rJNI76wrMuvkxl2DzsmL8vVhnjonVt87a5PTdOlZDipc7FU5hjX7x5aGBxVQzdPgLG5TCrzyfKM4KXrVHcjVLnPler8Zz9zGytT33NFsdcZhaUhkOWvbL61OthtuXz+CDa4f5j/o2Y2CWqSBoJ6obGXqPkW0dne6e4qBgBYAc3xypjK/dU2XheOa1FGXWiA12nWJW2YUlkcf/ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39860400002)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: y5Fm0/hNh+k2b1LXSyQV3EI5uNUJnhiVbSjl4lgPe7EE4G/J71py9SW6mksKmkZ6QEhjO/lp1Ds+Mq2x9mVE9lCy6im67Ti+1BzNk9cXjg5mZetGVrVQfkFHj2lYWZ/hFDWKFjjZUyDW/CCeZ3tFpTPmV0aZc81ePmy+yaU6pRNTBvcwFCbC67hJE4sBUxUSuGbEypajcvx6at40mYlD6QE+sGILV62vTP4Simyi5KRL6q2EOYN/QPeVcPf6zMzJ0qTyaA6qdJGUpQ/Ackb1/waRgV1tyeDKerZkHT4/N10Xnl/OB44U8QrEqw8TlIodM2VBdnZURliJytBAJCCrdFo3TztfmMn/3SyU7HBr3bkwMxRU2sLU1PvNkaTjDrUjkiFsq1+Ia1sewNSMvpNL5UIexBioMz98FRDKf0K0uPKB2lOYyJ8yfHTWwOiLcF5xH++elTC1VqWLnHhgSCeFXCGVZielIRAEhC+bKznXhwT0z9sP2A5yNGUqBIGzJyNQXt4EfkiEWTYt2QeRoiGIcCwg3afUjY9jDQjviplQIZNfjSqYhcFKAjb/FMYLJAk7v4tilKgQQ1X8K/+L1qWN7RW2H8F0cMGHE059VdtiYtq2DeCbc3UVVoP+T+FXaEqP3AG5x1R/proql29r7Krt1Zl+UGcatXWQOOm6FnV/htKb5eZ5L9xiH2BgmrAa1mdLUF5FzFDUq8431ZTlfqP+/Ps5NPk41VZMAio7csRmlUNbazV6l/GtpYlOtNMJfPJl1J1Xt6DWGnHohjTeuHG/6AjCYBFR07oMXE1cqRLKOq2OJ8GUIRg7jWg3JPWz8N94Gposmh/ZMq1uPUDRCjXlyLcdsBJndiH+Eh6FirrzeC9wDhyBt49letTdRCSGq2YiR0saWSpgHHfnKKVu5tobLgblRKqw3wJwmcak3ChlXIg91hOXf9kdapax9GPBjjAI6cVo4s4feAMG90xqJvuJjW2mBzJF8jkhv3yXjEIY3JfjXESyqd+YqFjnJjDHDdqYAjNh+fM6sX9/fIkjm3TMQ05qbQ/40CNIw1lW5+PLevDGSgxIm1iz3yJWBB4POY19S7obxlewQ2SLl8Dtk55alc2Skyu5nFVSoa1hrnRK8K7mpWGkQ+AqUhOREUszwoszGK+HGeqTddDpSPlUJPP5zWZkQ1iM8FUT4fbln6/A6yxWDOlLPC5Xb2ck3+IMZnmdddFjTpV7vBNzSMcvbV1+tGlL3B3DUjXTIeWPZgdbMUyTL00AP+MXN6Qp7BNEEHnrJP2AgYXOsNfVD//I3VYzmL9iEBqVfm7AhJTWFdQmCm29wP43rgxKIu7E8vcS8jaxGlvKDrqPsH86MTFcG1vF8CLT7ZR/s3Kr5LW1Ro6ROVVDBZscSlFTuh5dGgFyei57dHI7jlDbkWle69W/6XIbcqFHEnAAPMD4PyUuP8ebJuuG0Ih8NT8g1GIgwIKnoh277mr0sXcEB/6trFvhrrQhreMnAFvNWfMl3yF0fi0AWja/Rtxp86vK07veC7tgTSj3RFxD+IaccC4/zt/MTyqLEg5m6oPRlhVCa+LxB7RzUbuDDltLMe/2ZXzM8Gg51MWAs/sZoadaiQafbaIQ4fc4qg== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e48ae1d-f4c7-4e2e-5a02-08db9e94bcaf X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:09:46.2654 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3a5bRDt9PWrKWmvlxJ1j0gf8MUZwii8KCeyTAD+1IaQ8SutED5DUKKZejCsnJAUPeYiMByvFIkEuSxAfut+psw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6400 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:09:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186144 This change adds a new variable that defines where feeds should be created when building a packagefeed. A feed location for each package type is also added to allow multiple package type feeds to be created in parallel. The location for feeds is ${DEPLOY_DIR}/feeds/ and each package type will further drill down to ${DEPLOY_DIR_FEED}//${PN} where PN is the name of the feed. Signed-off-by: Charlie Johnston --- meta/conf/bitbake.conf | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 8daaaad615..381ca7f3e8 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -450,6 +450,10 @@ DEPLOY_DIR_RPM = "${DEPLOY_DIR}/rpm" DEPLOY_DIR_DEB = "${DEPLOY_DIR}/deb" DEPLOY_DIR_IMAGE ?= "${DEPLOY_DIR}/images/${MACHINE}" DEPLOY_DIR_TOOLS = "${DEPLOY_DIR}/tools" +DEPLOY_DIR_FEED ?= "${DEPLOY_DIR}/feeds/" +DEPLOY_DIR_FEED_IPK = "${DEPLOY_DIR_FEED}/ipk/${PN}" +DEPLOY_DIR_FEED_RPM = "${DEPLOY_DIR_FEED}/rpm/${PN}" +DEPLOY_DIR_FEED_DEB = "${DEPLOY_DIR_FEED}/deb/${PN}" PKGDATA_DIR = "${TMPDIR}/pkgdata/${MACHINE}" PKGDATA_DIR_SDK = "${TMPDIR}/pkgdata/${SDK_SYS}" From patchwork Wed Aug 16 20:08:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F67BC25B75 for ; Wed, 16 Aug 2023 20:10:01 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (NAM04-BN8-obe.outbound.protection.outlook.com [40.107.100.111]) by mx.groups.io with SMTP id smtpd.web11.170827.1692216592984145053 for ; Wed, 16 Aug 2023 13:09:53 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=O7NrnBJ9; spf=pass (domain: ni.com, ip: 40.107.100.111, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lUKWnoHo1ZyNxvbQomhCLn6OpXpZJcgFpvkeA1BcVK2WXXcVlFMt5P2IvwoXBIOZzHIamBcCX/I+uH4FJna1ApVQcILs0DbeGm8aTPcngmSqvVgiFpfgX+ZhjdlYJi9Fx+Jgd9EKj7XiDiQo49dMdm61GdGooMDZiUp6CJuQlOj3optNt2DzALIAxq2ChCvln3WkIIt5WbM7ocNdWJHRPxtvgcTR0oyhr/8/Y4vvspIc9dM4VnQCL6QXBR6pVXu7THRjb8CXEOeB4rN8B5PhFh7mugnBSj2Mjh3s7g2ZcPXseIs3oPIgI9erYweWHZjx0iJqMUX8hsUmCmBrvQFXEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ilVtkFbp8povxKtNiQf88QEAzl59bqlCMbhe29lE9xE=; b=ly7fwfWwUs5QXqkQlqn9dfChUvpisNzLxNz3KUs6leE6whjwZDyAb3t6/Z7Yj1umJoFW8F8T/P0mM4TUsYGr8pS8ZK2BCVqC976k87pIgQZRR/KT+f//m58pANto8EpuAJvhTfuChXXSo049klROjOxUGkqM0PyiPeNnR90HfDDV9VTE8g2EmP178ob9V34uU/9cp6IzlKYcBdwXNuC0phXWA/ECOBCucZosbjsNCDrFklJwIOx4VoeN7ZRkqHxscQrHJM16CrhxbireT3jebVSaWE7MFQMwoMimPQIrMTMMsvTVsuYeTqOJ3Am526M8Mqr+SNKNEFq/yvTDBm1aag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ilVtkFbp8povxKtNiQf88QEAzl59bqlCMbhe29lE9xE=; b=O7NrnBJ9oD5AeQkcSTr27UuBAvC2ap8RFeesPtVbMqjy9ck8ZRDJeoJvKAcvwrdWVOC6oNSBDuF0Vgp+nD+Vmxj1ZOEL4ZqzLBRZWv6teWcPdieFjrLpRMEfObg6rz/HO17qo5rvFIRWsJso60L3ENZIj/vKqmL2xm/9rlRknO0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by MN2PR04MB6400.namprd04.prod.outlook.com (2603:10b6:208:1a8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.26; Wed, 16 Aug 2023 20:09:51 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:09:50 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 2/7] package_manager: Add feed support to generate_index_files. Date: Wed, 16 Aug 2023 15:08:32 -0500 Message-ID: <20230816200931.2577181-3-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: DM6PR04CA0026.namprd04.prod.outlook.com (2603:10b6:5:334::31) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|MN2PR04MB6400:EE_ X-MS-Office365-Filtering-Correlation-Id: a6dfd6b8-42f7-40fd-66c1-08db9e94bf73 x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GlzBvcPcLn46dDxzl3Ob6RVixQhBwWg6GlGldADwB0IKdGLUxPu+YPFvCY/KxLAtkBCMpCKcAdtYPCN+RZJvVlvb3E5aTk0MMOfXJtjFjWLnYhxeCjYdNlTdQ3XWhhulg8xgSfRlzVA6cSP8LWphsGB82V1StaHkPhat1T5TL/V5FRoUlrfKRyccSaunac46JHdKsLgQAhLO0U6sE9IK91LshJov9qzyngzyBNQg08k6fz8bGXnkSPbTtg2es0UZevqXJSjfTGoHzcuRkYYvchpoH651pjK7a4x+qIfziGTVwM4RRugxhk2OaziiC4nrsD1oQN0uFRhUhfAANu0i1A/4EMoWkeCp+vG6e1QnWithB13KbeKjzsVkJuiRoxd7haxK87RaY50QwFcol2Ux7Ka22tOAouIvfTT5nY/guqpg3653VsolbPMMBkqJXWvldcm9SCOWITtNfQ5rixw5jNAK3Lxj58dTydJxnZI/+52pNIV4aJu9ynAoSfz84SS4E5DGfmiJXPJmvOKWnJ1ipQSgC+tof65TQHcrEMp1axilOHMOzQ7E4LYcqHPy+BVD X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39860400002)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(83380400001)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1qBknhT1BM103dqZ5exP9kDjejEYPoazfy8FcZDGxFo7GfEcFYnQ5rXIxIDY0nuZ85DhsHODKuC8IzaSJLfPAoxKDo2sToWbQGkh+OQ+ChjOzAypPYlcIk2DC+7BD0jxp72j99WT0PjPrxpRmxZB2YbFXA4Gi75XYXsbVCT7JNquCZ1oD4Z5F/iLDIry5JhWg+0VaXsgkiTVFMqb/5l8mvCY9OQnaFbgesvEkt1jl7gYveK+Fg6TZW0QShzpSmF7YagDRVQqDNkWIxRxZsJ2hLRWu/VLPNNHzg5RzBqWFi0jLQJZKgTgn7e7H2x/7xMyII2OuvRkrTjG5XgRkZN8t4ElD5d0wj9YGUSn+BCO5GXJQpWyo6NH280ItetNWWT7juvvIItnHDChXXee1DvF1SkGvSfxaP7+v+Iep+UaLvyUp+uUNj3W48QL4BDoN78arhTUD96UIyTUBS955O5MZuDKvlo7jywTHIT2nLK7wuxgr3jnJE0XZZ0HhUgwZltNMnuuCHeTCGTDf/68JvtK0NxQ8Y4UusiIZj5NtNGjeoUw8KkMpVKYRIqoFJUlsM+nOLSzXmXVdtxdDL7qHK3oLxnVtVQ7PzmwTLEtgLly/oMh6VdEXuNajdAA3WqebOnAcQuPUqm93160nXh1MksEzocTImeqiFqBdKFs5jrWXybSjJ8uhqG07o+ITJbHrD1LvUavQVUo6wjUL9ORLNwyPwwSpjWupIPjFtteateq57pbe0XxBczm9ZCA4XVYFyYiVFKWTX3pt+Iy6o+SM3+exTOvH7lLO/pqWR+IZbrhGb9ZlaB+d8SoAO7+zc7vDOahFtEDwIgUkJlHsI60EPEQPzJe0zkWixUGRLocsOBvvgDHOTSqWjbzr8c/Y7ZiyZPzUUs594E9c5v+xuYyhG583G4+wmhju9BdjgQ0ZXiolB7yNxcwWrP8kXcX72VE0OyYtYZAMQpLbIK6pQrXnlet4LMIdC88cp9lzotw3OIzwb/uuc2cRQ2FdT4MxOrLf88GVO1Ko4yDjJADZnBoUWLE7JKHDK5buaBskEQ86FZza5pbM5a7KLT24L6ZqKzBsC4UnhgkBY7EcptiIrjlTnX3NjA3ISeqIo9jmcWbmoFlw3vVP1SK4hsoLga7t6VwjlIHpFkGM90B/ckAw9oaLSFAMfOVaXcZM/VVyqkOAzDMrIPtFgJUdhCZbQbAlyIIXIhPjaGcANm4lWMkmjWOslAz7hpSHA/YHQJ50ORpSYKqQKm5JA4osCQzMFYvJNqXE+liEfPee+meu2twBksJB7SxDxnQ/NrBs5E4z/IEXhh4hYEw8rTMmbex6KzcN57NvvgaLDMBI4uzQbxx9C4grhP/yDEs6+73m9D7+YjqrcNhiI0mRnw2bEnmSckCo4pz0bGOagYGURQpAOu1b4Zx1EX3ADuFW3GqNftb+pSDE1K5cG9fRmplBfEUvJ2HVpY5aEANKupFL8wg+9lDmMUuIfwdSB4r0Vph3WfGz51/6zqZtHTkYUBe+4fXj9+PsJdyeS7NkCs3216i7lt1XqLRUIX4gpRAepuW5ppiyETdEUngW3As9/B8aw2ifJckkiBaLlyNf5zg3j9wLqRYc61NukzKqw== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: a6dfd6b8-42f7-40fd-66c1-08db9e94bf73 X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:09:50.8668 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2Mrap1ETVtWO07BN5MdL4K3dt5m6j/h+/RQlViDqEhX/zGedha1AZHK9y1kt3eLlG9FS7xbVCrBtKiNpbuDvhQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6400 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:10:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186145 Currently, the generate_index_files function only handles the creation of index files in the DEPLOY_DIR_ directories. This change adds an optional isFeed input that will instead point the index generation at a package specific feed directory. If no feedname is specified, the original behavior persists and the index is created in the DEPLOY_DIR_ directory. The directory for index creation when isFeed is true will be DEPLOY_DIR_FEED_. Signed-off-by: Charlie Johnston --- meta/lib/oe/package_manager/__init__.py | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/meta/lib/oe/package_manager/__init__.py b/meta/lib/oe/package_manager/__init__.py index 0c313190cf..af4254caf5 100644 --- a/meta/lib/oe/package_manager/__init__.py +++ b/meta/lib/oe/package_manager/__init__.py @@ -533,27 +533,29 @@ def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencie raise -def generate_index_files(d): +def generate_index_files(d, isFeed = False): from oe.package_manager.rpm import RpmSubdirIndexer from oe.package_manager.ipk import OpkgIndexer from oe.package_manager.deb import DpkgIndexer classes = d.getVar('PACKAGE_CLASSES').replace("package_", "").split() - indexer_map = { - "rpm": (RpmSubdirIndexer, d.getVar('DEPLOY_DIR_RPM')), - "ipk": (OpkgIndexer, d.getVar('DEPLOY_DIR_IPK')), - "deb": (DpkgIndexer, d.getVar('DEPLOY_DIR_DEB')) + pkg_class_map = { + "rpm": { 'indexer': RpmSubdirIndexer, 'pkgdir': d.getVar('DEPLOY_DIR_RPM'), 'feedir': d.getVar('DEPLOY_DIR_FEED_RPM')}, + "ipk": { 'indexer': OpkgIndexer, 'pkgdir': d.getVar('DEPLOY_DIR_IPK'), 'feedir': d.getVar('DEPLOY_DIR_FEED_IPK')}, + "deb": { 'indexer': DpkgIndexer, 'pkgdir': d.getVar('DEPLOY_DIR_DEB'), 'feedir': d.getVar('DEPLOY_DIR_FEED_DEB')} } result = None for pkg_class in classes: - if not pkg_class in indexer_map: + if not pkg_class in pkg_class_map: continue - if os.path.exists(indexer_map[pkg_class][1]): - result = indexer_map[pkg_class][0](d, indexer_map[pkg_class][1]).write_index() + pkgcfg = pkg_class_map[pkg_class] + feedpath = pkgcfg['feedir'] if isFeed else pkgcfg['pkgdir'] + if os.path.exists(feedpath): + result = pkgcfg['indexer'](d, feedpath).write_index() if result is not None: bb.fatal(result) From patchwork Wed Aug 16 20:08:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D7B5C25B73 for ; Wed, 16 Aug 2023 20:10:01 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (NAM04-BN8-obe.outbound.protection.outlook.com [40.107.100.99]) by mx.groups.io with SMTP id smtpd.web11.170828.1692216595388184136 for ; Wed, 16 Aug 2023 13:09:55 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=MO2FOCse; spf=pass (domain: ni.com, ip: 40.107.100.99, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HyQyi/1bqks1crUq34UDHQ0kKT1DWkvc/7x0qAyIf64dwZ/pClAk/Y86Tz3YvTGnEIJMHOsgmRlg1o94rsIzo7c/j9eUQUcnnSYU0QEuvDlh1k9RVi15Xc643cTZpwKAQinIvSQacd+16DjWk99cJNWHzK+EZrWdQOiwRR9iEkXFUvrzkTqTRWS9IT6bktuikpMjOP2zOTNW/myXBCfzcdiN/qvLj9wrLwTrc+kySTYXA8VBJH351EVQai5TNeWie8z3l1a/3Fd7mYB4hg4pI8bvrZadw0w24iu4BFqS4n7+o/WYK3YHx5ApWGERI4Q3XPH/+oo3zRLF9AF0AdpUkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TuPwA88eIXUpOOgCZaOYW5rrr+ec8kp5dtlBDAw1nJU=; b=cGYViVKVNQ0j9rHHAGf9z3O+xj/E3TKxXcT3vFotNAGtvI71//+/enYQkl6YEwU+qFmmZFwG5kef2Yzbd7ySF9+qfmz7NkAzVFU6r1s89d8BGX6L6S5qGi8yWFTd8ZZt4cB/I7QWYHa8IHEAcYGvT39lXrcRdgWulS/39SNOdOMc/byLOnKSPCGFe3p4JTSnkhapE8lgmZ2hBwsiywS/+MjG6PZttzOF1D4TVhMu0r9UyNrng3/I+jEgNpe9TIi7iGCVKRZbG2fLSl1t5mNECBVFHatMpqnjMD4BLEYjH6NHUCqDRCF8zEZcg9t5R2clU1SIff1noLK/9oR68guNSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TuPwA88eIXUpOOgCZaOYW5rrr+ec8kp5dtlBDAw1nJU=; b=MO2FOCseNPrOX+SdeH0Wp5Ap+JOkRWZvu2hW2+S1ie1tKMH4ijG4JokaWywVBdXrrr/WKuYP/Z3EkqxKaCZIOoc13M7fEf4WseitJgIVLefFzOHRPPRIr1iVG4S1O7qyxjEfn29aoLG6BIHleYC6Sr9eXU0Nj3h1HEuHW52vL58= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by MN2PR04MB6400.namprd04.prod.outlook.com (2603:10b6:208:1a8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.26; Wed, 16 Aug 2023 20:09:53 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:09:53 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 3/7] package_manager: Add _find_task_pkg_deps helper method. Date: Wed, 16 Aug 2023 15:08:33 -0500 Message-ID: <20230816200931.2577181-4-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: SA9PR13CA0174.namprd13.prod.outlook.com (2603:10b6:806:28::29) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|MN2PR04MB6400:EE_ X-MS-Office365-Filtering-Correlation-Id: c8ab97bc-819d-4e22-4a3f-08db9e94c125 x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Rwdsc1oS0FxSAxbDsvJFELx30igGOOJXy3zMJFbcyzKUyflq5aVyKDRXL8P8HA3KMNOeWxiY6PWLo4wNJCaHuRIndv+W/XK9wjjrfyQKKQf9kJ8aZiZVfYfG0rKCLS2PDW2V66jekhQq6ULw1YLEXiEU58XSML/vW5BWin4SAXRT4ab5qASOh/YF31kHkNwHH0xDQHxATMdcuyPAGgISot57elhLMZIpuoWTnouUeOC/gfx2oXGlvnd5mXW0jKjY/YRoeqVnu1AkkZelPxPiZg4AiAdCBhnrN920F4jXH7qalwyjfhxBGpG2DwSUpJ96IlpoRG30AEAc6hjDImeL62J0nndEfPn9+Nux9xs6YnaPABtzsTIV5UXLL5d8XmjCnyNaolE34pinyp5/XHOZ1ZpYAS3ISmveYueiBF1C6pfkCDe/IWpxyUzcHR0H3uT0ToDDkUz54DN7kmzUTgwCPJHgzirlvmijkLef7ig6b6kMZ5Ek5CZNFe0UVSmqYvuTqhF7bAJdC5cYftwLUkuI8Azv8yQAeT88m5c+w+Oa1Tj+3KTAMb79oc3tTPwn5VDB X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39860400002)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(83380400001)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dTBQRIjO3r7zuaMGYv5jieUtkQH1C/PnHMug/261XRrZnbJ5lec7fL7+JGIfw7c8aVjhcbRps6O59gI8aBC2tj3LG1X9XbBzuZCbM2+wPqy+149Lr54pi+g/Ht/SPc9X5NzzTpSDP/RL/gXVQfmF2OmaZlgag06pHGJs3NyyLf7KPFxMcDJXOxqUgpVxZedqAEuc5ps8OAv/zTvd/MAAuilaUqDQ0evUz2/VpvSIJJkMsHDFSwOt1kUpaCyWRvoljf5N+d3fxCjVTRH0XgQE0I1ncohYJXsnd14PSAM8gIDvqZdbYsFOBS1DUKnGft73Ce/F29CeCVRAgpmr1+byRj8PCWV0p7CFLunTvdTMi6uSZvAnRezqcqBS8o0LZYmG4qpNAhq4J28VfuKiqfPD+hYwh/UNZIksUTxQxUmbkoxy7d7DE1qvcvZWJE3znCxFenMVJTqICa8Cp/O7BbDQxhZ2u1f/8ob3ktgO2YTizumRvI1brsh2HOzazROxeZtP9laU8BB5F7Qwm0CYSUGKqqE/8GghgDg+jweokL3JJbYO5X0jKNfopdnKdCPNN/a88SO/9tTpcIIuLOU+mxT6HngRQ7+HQ4to5gCWM6gpFVlp+fcaZiIHzk63fc4FT1GYqrv5AKgrcMRGhscMwMhyW2m4r15lVtJ20kBUWo05DhYn/jmFi7zGspX87IxboYLyuffDkODHAUxNuYvmpO1PD0Zz/msTWBLzPRZsw0qUYu4O8YfuKvnipq9fStvoxhnpQyT5XkgdWswYkY7MUnNoHm88PoDi6AEXB+uq0SbT8NpZmEuFBg8sKjVtTGtqkkHxYacvS9uiOdgRpe/aeMh8drnXqOVgjZLk1ue+gpmv577VJvYwEUdW/wCtOyGL0CdmJSxxwjmHmXZCNgubsMDUfHfcX0/CpRJSP0Ke7zXhOYjVU1bWU1cqt8JG7wgJFP19MTD+hE0cjSKFrHJNav1Wtc4oeLgh0lPo/Ij+AHEVTlLSZaZIMyYa3iIgankRIYPQt1iQ7iIfDvIqzFpZUNr11hneVwU0mgCBfbW1mTSm8kau1fAccveUAt5MNvrAwzrBtoYsV/YA/N8YKYEBbFtikpiDWQ7BUtoDLGYguozu+Jur9KcaTbuDlgU0e1wIUeqnvktMwYNUQgXvRgEA+Qf0i/cVB7kBL9f2LhAwai+PKQpOcrhVdzN39kYtc1Gw/p2Q6OIs0v6XWysL/9s4ZkAPYNSqUlazHTSX2gkMw9s4dLm5lzowIiuOyrHqfe8+qD5Q4/haZl2TzxlPfUmn4ZKIfFAkj8dhMNJzYJXtzSzq6tgSGDwOQG9HlylBZwiW1IUWdmWxnh3wDBMNS83pOu9uipgTvUAbnPOl0j8M92HExK0dk49anxY97Wep5tapL0YY3ujJfREhOzOF7P6WxVoHEqtNTPKz8kUx9cnVoApniNiOLf/ZPfy6o+kZIbhmEcOS/Ppjgm09fvHbeyXbkLgsUY+aAd/4AuIqhTx8qP2UPTldW0uYFG9Btlr24rdGwNG9IH2Yw/GTG3xd1FdE2wIIJ8BYAtaKbV6hpDHTwphk1dJH7/YfBv9a6fHgdQ53pOeHb8AhxaVXLq580C/KhT+Kyw== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: c8ab97bc-819d-4e22-4a3f-08db9e94c125 X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:09:53.7733 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PCdV7InRMJ6V4YwZPAVAPzBeuC8lnbV0K4UQ+X4BBXhhI5aftXuVRsKeXZsrZvpsv4XPbxXZi4lshVVUMyIHmQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR04MB6400 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:10:01 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186146 To make the logic from create_feed_dir reusable, this change splits the logic used to traverse the package dependencies into a helper function. Additionally, the logic used to find the initial task was updated. Signed-off-by: Charlie Johnston --- meta/lib/oe/package_manager/__init__.py | 51 +++++++++++++------------ 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/meta/lib/oe/package_manager/__init__.py b/meta/lib/oe/package_manager/__init__.py index af4254caf5..10376dd9cd 100644 --- a/meta/lib/oe/package_manager/__init__.py +++ b/meta/lib/oe/package_manager/__init__.py @@ -473,31 +473,7 @@ def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencie oe.path.symlink(deploydir, subrepo_dir, True) return - start = None - for dep in taskdepdata: - data = taskdepdata[dep] - if data[1] == mytaskname and data[0] == pn: - start = dep - break - if start is None: - bb.fatal("Couldn't find ourself in BB_TASKDEPDATA?") - pkgdeps = set() - start = [start] - seen = set(start) - # Support direct dependencies (do_rootfs -> do_package_write_X) - # or indirect dependencies within PN (do_populate_sdk_ext -> do_rootfs -> do_package_write_X) - while start: - next = [] - for dep2 in start: - for dep in taskdepdata[dep2][3]: - if taskdepdata[dep][0] != pn: - if "do_" + taskname in dep: - pkgdeps.add(dep) - elif dep not in seen: - next.append(dep) - seen.add(dep) - start = next - + pkgdeps = _find_task_pkg_deps(pn, taskdepdata, mytaskname, taskname) for dep in pkgdeps: c = taskdepdata[dep][0] manifest, d2 = oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname, d, multilibs) @@ -533,6 +509,31 @@ def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencie raise +def _find_task_pkg_deps(pn, taskdepdata, mytaskname, taskname): + start_task = next((dep for dep, data in taskdepdata.items() + if data[1] == mytaskname and data[0] == pn), None) + if start_task is None: + bb.fatal("Couldn't find %s:%s in BB_TASKDEPDATA?" % (pn, mytaskname)) + pkgdeps = set() + tasks = [start_task] + seen = set(start_task) + # Support direct dependencies (do_rootfs -> do_package_write_X) + # or indirect dependencies within PN (do_populate_sdk_ext -> do_rootfs -> do_package_write_X) + while tasks: + new_tasks = [] + for task in tasks: + deps = taskdepdata[task][3] + for dep in deps: + if taskdepdata[dep][0] != pn: + if "do_" + taskname in dep: + pkgdeps.add(dep) + elif dep not in seen: + new_tasks.append(dep) + seen.add(dep) + tasks = new_tasks + return pkgdeps + + def generate_index_files(d, isFeed = False): from oe.package_manager.rpm import RpmSubdirIndexer from oe.package_manager.ipk import OpkgIndexer From patchwork Wed Aug 16 20:08:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14BE4C41513 for ; Wed, 16 Aug 2023 20:10:11 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.132]) by mx.groups.io with SMTP id smtpd.web11.170831.1692216602317089389 for ; Wed, 16 Aug 2023 13:10:02 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=fXoEnNEl; spf=pass (domain: ni.com, ip: 40.107.237.132, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cfh/GujGYT2sB06mzR2kMIoNa8buCtI+VA4ryf36dQ4wQFC4dWO5E4npsyof80k6tZmBJZ3ZeeEYnICHSO/Fluqi9W3p301ktVZkXCKuv4FwTSYS/ZBfzBrYGl18UWixqH2HFq9CWXoJYEPtg/ImdLVWG/pFa6tTGYqwRvdrl105AA8s8+UCdXiUxGMOfHrR09CLMJRXRO1XPDKbzhFSPx+8PlJ6y0S7d9RjA1LFrkhGcFrVVgrz/uZdV/OShPb0efPq8icENMWQpSVxLzIIa+Ty2XqOxmrh34h5TiAtuT38krdrLWf+6siM3v3gbtKfMRBCC5PKCEPFFg1mi8RZuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uRDSgsXOM6mYi/9X1SJY2pJUML4f21GWOnaQastIA3k=; b=lLXb+tkF5nHz3j4EYU262jA+Xe+xxUBoklXEmZb1OtxK/AGjIlIhuwtuWtqm5wIL1CPQLb97P7N0VhJuHW1CbCLpMGnoAEHS8dRli2VjQ82wkj94BY1YZR1xQHbqCHWSQeN6HT3WLbed1JxA6iucmiIjxHvYSmgRGtvySgjMrT/dms1ZyUo/M6rhmxhJVk3FNcvQ0Fh5I76tW5QqVpnSXgwALIQu5DPzwIsBFD3IxzbHrn5lB2ApQZys5AbOe5EiQb8lnBCtW4M92yeyMdQCezSRmGIuM1fwKaPu/skJS+9JIi0LtXXkG1rzZiRXdIXsiT4umZGsYJLMsw+MBVWg6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uRDSgsXOM6mYi/9X1SJY2pJUML4f21GWOnaQastIA3k=; b=fXoEnNElMQH8KYYJeoNRHQJVZ4NfGfn9eeq5qRZfxMUX42kCegQJMb8AibSn18NGdHsA4C8SV7Y3/P0QvK5yMBhdqo/ziXJHAmiB000SMyAxgRZ61bgDfFKW+s97Cfm3FK27a18bwH8rYjMNNy/SBD3z6CInrQMOdkW5zazJ5Ec= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by BY5PR04MB6963.namprd04.prod.outlook.com (2603:10b6:a03:228::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.29; Wed, 16 Aug 2023 20:09:57 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:09:56 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 4/7] package_manager: Add generate_feed_dirs method. Date: Wed, 16 Aug 2023 15:08:34 -0500 Message-ID: <20230816200931.2577181-5-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: SN6PR08CA0027.namprd08.prod.outlook.com (2603:10b6:805:66::40) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|BY5PR04MB6963:EE_ X-MS-Office365-Filtering-Correlation-Id: b007fc11-f734-488a-b70d-08db9e94c2e0 x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YpkjUiQ5dEhSD00T+bbv95o1gwZ4doeXkk1nleaOKtjWXJUrl2SHHlet1Fpzn4ofGivTGKlqhpj08D/GWLWGhiEKUDpjQd7Z/l89z/k+eZBcXJeQxiSVIJOG8Y/VOU7N1b4cvytu6xhqIDA7BrDcyCKpX+pNnEFBJdsp2RYDlR0d7DlBL5Hq07x2KgGDt0Ncq1ZwAihXxLR0Y29hp3sALxtWRRigfs4wgZ4lIZbpLUnwPm+ACIPrh5PWoM8VjwMyLfU9bMJD4ocIAsi5LopmzY+ECMmxarw0qhw0ZlORML5RSs2XGk43wm50pWGk8cURc7CWTljsD/h7BaCCIeykNpwiZthbTT2UlJTO2qMRQnrwVXC6XQUKIabEvhfMX7xv8N5LbBFz4BCQKeEvTIs6mlXhL2gAKZg6ECfrsJ4+0Fhyn65Pwo9h2OD94AfOfCL5qOQUfjC0pbrkJx7bGMCqMMr5psxO43y+B9nznGKM3suS0Oc6mowtUCHRL4GiQNjMtOeoOmnOlTICxwr95zZbHC7loVa8Jm68ISIQvCuGRXAGy+ClW16VVBgQ22R8YdVz X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39850400004)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(83380400001)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1TRo3qAKtNUcSYv2NNWIK1O5I6K75dLbW6bCN2vH5RQUls86OuzjOz7YbMnIMb4yeTWauI2lInNBSnXpkLZOSVRZfcDH7lt1v8wJuWEO7LpOgZ69ct+tguGbk7E25bM8rxZaXuMmdeFXGYXEGEEU3SKkWc0fGqsiirOCIFzi/ZyaVU7uFkg2/SC8IBNphmexO/MUDaa/k5W+8bzUrdBoX4HoqxPX9tKplLI02SL0utlWq9uhVng27sL5mKhb3/yudKXAgxzEBnDBL3DGV/dg07nQJdAdXcQweswt3fBsGlQwj4Ve0Pe8+sAxj4N6TXZfn8pO9jANIDCfaQBAc9GejojCIDSdW63z30uuH0StQOFKR2tVrpmZVXlzi8yDWUsb6AfwHLlDbyNJZYLNRWsPdBJiN5kxtsFFPBvzWcYrxqL017nAwTnZDSp/6p3/MRqj7X1qsFXa2M/rXAlZKe1XuMAf8qISHEmKKPYCFQNj7oCfrjbp5E2ClxJEM1w+aKWEMLFRke/HuAaBFNalWxRwHWkmKbCJrFJHmy8lUl8NJ6zCaFDyax7WtfuuT2LFkAYwJUMQhnwJwaJ/1gZpB9Od2VudSfIGJgmV0H+L73hhXaPO8EZZ8ckvopChe0IoSgxQRLaRaBvpe3vXsNRi5OPtKKiV6HNFgkqnE5xBNY+2qtvUfKntUHmkI3vfjf9lustwgrZQlPkqhPfePMxCkTIy8oQFk4sw3aZyK9I3+ID8fO4e+2kSL7B+RG+jjoUwpYO7ukoby85g6knzIg6s7w7ad+VnrmhdE+FhADVkemtgZO/vpW2owYCOoCUU1gTY0H0vBGr4aqOcd2viFqqJiiNkc3x5uE3eV4K5GRVJTpjVV9P6mn0M+qOuKn9qNKcfS9i3bzQdA6pWndV1unny2JhVcEEwEYZtt7IUDnombK700zD1WXNyZlVwQO05HOEYOfrQyzX/MWoNRuTSPNV9h2VoI4S+zjf9y3CMIn8gwi+x7+fR6+t52EGtFrJmKk4RGYg5WSAUmLLFtPFw9nSbduzpZKBx1TnNSX4dKVoPuvZmuAdaS4eMS1xD+GolTlBiT/DZeFXZ+QsDrE3jFLOLWZUeulGobsROEGD6s/zZSZqclqto6GMcdmxgsNj0pNY1wXChpakSqv3z825iOR3Iyvtbm7oga1g9R2ztqvgX7rtZLW8cm/Hr143JmOxcMNitEZw25/ApbDEjMk4S3UasBWdZBurxL7IEOTBYbWYy/L++7kvNX6Xb7NLHCYKLIxjWfDldXuPQgGrwzAur1bWFTTr7f/S/Zwq9Qt2pZy+AjsQHv4bGfoE1QRYqPXgkN78Nb8krrkilqo3JM8vllUaT12jxEd068FJo3z6RZUUnK6fan3A73buXk0ADxuJUsA/YdY6ZeUnt14+x+IOi09r3rDd4kY9t3+LGlhMKzA0Apgadg0+aEQBRL12m/Z+v513yxUIaOKI/SHKKjtoTW9A/qqw84ATdXiWu2KcWO5qGARM10UCbmjwvJl7aSj1o8XYOxKk3R17ZBUBozDnGoFEJDd0h5ow23+Kceu46Sa1u16grszZXIM9iWbyYiDvf+GrYJhn4CuejvpnXkA2acgxexYm2fQ== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: b007fc11-f734-488a-b70d-08db9e94c2e0 X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:09:56.6202 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VzgREpMqd0t5/XCS0Y2NUzQgI8O5jxd+gmIjV91pt8GzFqgHX+uyd/QOcJe5kagVF7Ne4cApr8zYcE1ZZBe8SQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR04MB6963 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:10:11 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186147 Add a generate_feed_dirs method that will call the create_packages_dir method for each package class supported in the build environment. The value of the PACKAGE_CLASSES determines which feed types are built. For each package type, the new method will determine the proper name for the package_write_ task, the directory to pull finished packages from, and the feed directory to copy them to. To support side-by-side feeds or situations where one feed is dependent on another, the create_packages_dir method has been updated to support an input called assumeprovidedfeeds which lists feeds whose packages and dependencies can be assumed to be provided in a separate feed. Those packages and dependencies will be excluded the feed directory for the new feed. Signed-off-by: Charlie Johnston --- meta/lib/oe/package_manager/__init__.py | 27 ++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/package_manager/__init__.py b/meta/lib/oe/package_manager/__init__.py index 10376dd9cd..e87ae6d8d7 100644 --- a/meta/lib/oe/package_manager/__init__.py +++ b/meta/lib/oe/package_manager/__init__.py @@ -449,7 +449,23 @@ class PackageManager(object, metaclass=ABCMeta): return res return _append(uris, base_paths) -def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencies): +def generate_feed_dirs(d, assumeprovidedfeeds): + classes = d.getVar('PACKAGE_CLASSES').replace("package_", "").split() + + pkg_class_map = { + "rpm": {"taskname": "package_write_rpm", "pkgdir": d.getVar('DEPLOY_DIR_RPM'), "feeddir": d.getVar('DEPLOY_DIR_FEED_RPM')}, + "ipk": {"taskname": "package_write_ipk", "pkgdir": d.getVar('DEPLOY_DIR_IPK'), "feeddir": d.getVar('DEPLOY_DIR_FEED_IPK')}, + "deb": {"taskname": "package_write_deb", "pkgdir": d.getVar('DEPLOY_DIR_DEB'), "feeddir": d.getVar('DEPLOY_DIR_FEED_DEB')} + } + + for pkg_class in classes: + if not pkg_class in pkg_class_map: + continue + + pkgcfg = pkg_class_map[pkg_class] + create_packages_dir(d, pkgcfg['feeddir'], pkgcfg['pkgdir'], pkgcfg['taskname'], True, assumeprovidedfeeds) + +def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencies, assumeprovidedfeeds = None): """ Go through our do_package_write_X dependencies and hardlink the packages we depend upon into the repo directory. This prevents us seeing other packages that may @@ -474,6 +490,15 @@ def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencie return pkgdeps = _find_task_pkg_deps(pn, taskdepdata, mytaskname, taskname) + + # Find any packages which might already be provided in a separate feed or repo + # and remove them to avoid duplicates. This assumes any dependencies of the packages + # are already met as well. + if assumeprovidedfeeds is not None: + for pkg_pn in assumeprovidedfeeds.split(): + provided_pkgdeps = _find_task_pkg_deps(pkg_pn, taskdepdata, mytaskname, taskname) + pkgdeps = pkgdeps.difference(provided_pkgdeps) + for dep in pkgdeps: c = taskdepdata[dep][0] manifest, d2 = oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname, d, multilibs) From patchwork Wed Aug 16 20:08:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22105C25B79 for ; Wed, 16 Aug 2023 20:10:11 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.132]) by mx.groups.io with SMTP id smtpd.web11.170831.1692216602317089389 for ; Wed, 16 Aug 2023 13:10:03 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=epH6qnb1; spf=pass (domain: ni.com, ip: 40.107.237.132, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W2s+TJ79heWC+q7NexsdjVA0FcHN9rg+73mPq25CeRol0h92iZs81i4br8K51lYUtR11nA6sv2wpXBeLmNssTYPp4PAOAQRcJ6m0fB+ZpwodEseovk//mWIoAI9eS9gePuywrw4dAqk8dZme2t1pV9G4ivNW8yiKNDm4eGPSbC85di3qlqy+iHWrfA9StQyn09mq5w55RXsxBvBfm+30xjzlPrvQpmPwrscfRnJBdZPaFToWdh7VJWXQEXqm3pbIsW4EpivMyKoenCiKFpvdK5CjdHecyKb+43Abo0CHVKC0LjekFzQgxBUB5s1LsgSwj6zXkxTeA0J21iJhnHXCxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tZRLn6ZJv4gTdR2Cjmhe9F0lB9WmudVLZUIdQDIqkLE=; b=Gw2DMVwBivuVDwUfkZxUT7/8I1hr1iNiM1S4Xwqka9HLfJJtNnJ4pqflTwcCAjp8FUz+udICRuA/HETsk/7uPMad7NCEaQlqbfhI+TbkHUFrm/t2KnFhMnvdJZtyhlLIJS5DFSFNJRDNlVfS33ROmXs+/+QZzOhUtgn9fYSJAPQ2PeEFbonGqH4ElBFRyVrjQ8yQyEDyTbAc5V/KzcI9fbTiB5wHtG6bGPnRwhBSF7XCfvoq7/k3UlAKHwjuHZyZJe0qaD5ViIeMQ0yLRkBpllcwFzHbAi76bNQr3o7Kh1MJ1J87Tkmthv0fiXy8L48cWx8B+eHS+agONgTb0D+2uA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tZRLn6ZJv4gTdR2Cjmhe9F0lB9WmudVLZUIdQDIqkLE=; b=epH6qnb1vnJ0XerQSOXu6UhxKCye+7MDxYA2jBlh3MJKpjXbmBblm+RbkKU/QU5/pzLxB566ryG/KHRgx9jyezgl0w665dVbei+g10j09frnhrIxtDpunRNfUsQ3mhEfqB0+k5JmcWsIfjadkCJju/16QpwyoOu8ae7xB2wJJ/s= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by BY5PR04MB6963.namprd04.prod.outlook.com (2603:10b6:a03:228::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.29; Wed, 16 Aug 2023 20:09:59 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:09:59 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 5/7] packagefeed.bbclass: Add new bbclass for building feeds. Date: Wed, 16 Aug 2023 15:08:35 -0500 Message-ID: <20230816200931.2577181-6-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: DM6PR03CA0099.namprd03.prod.outlook.com (2603:10b6:5:333::32) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|BY5PR04MB6963:EE_ X-MS-Office365-Filtering-Correlation-Id: 580ff72d-04f0-4814-e881-08db9e94c4a0 x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KOgCHZrbuT6P6TC4VUO+ZdoM9NSP2PYZcBbom5JmiN1PwslV1A/DPbao3u8ltE0XzcsXu/U91oQSl10rhAHRsv/5Nv3uWD/gF4EDN8Vv01BPO/Hkrpt0Q8G3qEmRAuq4SdQ6clyp1JrQb8QxCO7OlZ7TdJw6EMKMZ3vGT4JbXBjg2/A5K/4LHNoHQObeR+RRGuHF/vWDv8CIPgKQlbvx0E7qVOAZDsYVf79L/V0D86I9St13es8rQxUwgsfbcFqwgC7maamMx8neqS1f8RG+LzOWreeZzlSyXxB5BlQ3wFuYcOBkNYlTbdBzDorCkK4tZM3SNOnMqItcNlNY0IQVtSEcQX+v0UCqJctcCyv+qTTbobFoWAQUDds+6J06sOWG1Ocu2N1tR8a9vjDNCVCwHyAewuy6bwr5nCgCoLLTFpCpR5pxUDOuoNZ3mxmqcu+1aZG+6xb9IdlFyxmFf2rPetl/Q5xHJKpqma7J8G62J0kFRSnRKahD2ZKCJZWiN8B6pDV9rl3yoEg/4yD6vK4TUM5wmTKJ/sfmgPYHNw6pw1yzLhSM4207rFlxQI9DrG8X X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39850400004)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JLqa/+DuA4DbeHrbMf8DSdE2IHaDh2aLuvPuDCYjJrff1sJi4GnCMPQ7cQUx+yARd10WXPzI/aMc/WATPdrbC4ZyBWPmzcPTFq+NedpyPk5gqIHaCuB5MQ2R+tqJ6uMkpa7VOOjv94eCpLfsF6zTowOkOwZph7ZfZVIeJofRSGmi9tU4cjD+0OIAjxvEd8AGO9nddKYlhmYU3+wIq8EXklwv5b+MOr7tIiwSLbVXtMTtXnv7xEJNFX5rINAIxGnJbzSevAZCYL+F+T9zCK0EGYvWAPwvEZXYvhimIhzCTq2YPn7lFTudukKiLf4+q3lty21qakeDA7INLSYMwMQwINbPegW8Gxlb3qZFTfeUT7m2gVIiiWMXca0jnns2e/JMsnXuOsjApH9QYhlLg52I26jdAIMKXdeWGJn91MIwyepmh8OJvzFEfQ8JYns9JvGoyIpabQXHYLAqNg+vFDv+OEtWgUiv7fzo6kf78UC3It0fYNg/o2UIP3OGFAgoV7e3orIcrUaDWdhAVZsOD7TOwIzBQamQqLjEqbbvKxIM1NGyH9Eed3qhZNJmtxJlhuZ/kBNT3q/taVy1S+mURDjdRIvKpcaC1/TVuNg6URP3FUYOovtP9FM4LGjaibGVODW55tOf3ZGBqbe5sSAfiY6mgJQMG1/luUukYN44ugccFvZE6/qJbiq852jBxE8KDTa9lDGXIZocYLqKbcXaQ7NqDnVsN9wGl2ngsFpV5riv7/o5tms3uxYgTDYlvTp4zPWid3d1MMnelauimyT6TwTIyOIucA/ILlwlLWa521IByZvJR/Xg542CU88EVjkUJntniLl3oZAmslP74zXlG9f3QFluIwR/D2GpXQwU77C0WfuAwZ/Jt9y9k+wO9HXGPng25EgEsG8wrqQin2u5NgMmpJb8ZBWwH7mezjcr5LZ+HZ6n0zKbQY+jrAYS8AwlGyTU6Ecw9h4lOs8XC8SAPe8ulhtk2FpOnuHw7yXvB7d+exvSNiXx1iHwFGpU3hGgHhMrzLmkvoQKp5TyS1VJtlvfqdFXFSoLd/6Njye5sYq2SjsV+/AR9i2GtRkeqa19ZuRw95gDaWcTLJ9MEg6r9jP06FWNBgIqrXuHHAhxs5D+o7DB36md82z+TaCKLN4ln1ItqTcHJ6V7Xze8fVkjy1jWrltNvi7Bx1dN6RmejuhH/05iJRP5f2DkqufS25KKlIbHzHsZ7Sihxo6MnT0vx14I768ru9QoZYl2JcF8naEtMNyJLa2raGLPWTMXi5mIi9tIpmTZX761185K1LcwQsWq0XMSe7GVjjZIwgA76cv/UJwOGoyzBbSz+MZY3Dv/pTzJqLVIQsvwAkcEaA0TVLyKbGeQ0+1+BGxW58iX3/wMoNFWvoRySgEfUAcqDi0awHLdHe0txrquZa+DBhapTwDos5FOPiLvfhoiqyOpnTPPnN2PinFfK5MdVGaO6i111gFIxZt4IjoXMikQJp8uDju2jpgCSdhG97xsulwBpPe1u0u1QVvDp5RJQYxM2mKTe+GPyPtfxor7SM9D33vTa2EX9y4fu/vA6HFtvnyV47VabZN22zXt4GjIWMqKWy9+pOMSXEh8Dewrg8b4i4W/FV/AEA== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: 580ff72d-04f0-4814-e881-08db9e94c4a0 X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:09:59.5933 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QiMzpQmiXoCHObjsvF20SpO2r49fMBsAVDoSeHMMe5L9Z1n91eXU4P7sILJ+uDjlT90JcLFNMmgxpxzl3EN/9w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR04MB6963 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:10:11 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186148 Add a new bbclass that allows building a feed using the new oe.package_manager class. Additionally, there are packagefeed_ bbclasses to define package type specific configurations. The do_packagefeed task currently does no use SSTATE data and is set to always run via [nostamp] = "1". The variable FEED_DEPENDS is used to specify feeds that the packagefeed depends on and will be available side-by-side. This prevents duplicate packages in the two feeds. Signed-off-by: Charlie Johnston --- meta/classes-recipe/packagefeed.bbclass | 50 +++++++++++++++++++++ meta/classes-recipe/packagefeed_deb.bbclass | 8 ++++ meta/classes-recipe/packagefeed_ipk.bbclass | 8 ++++ meta/classes-recipe/packagefeed_rpm.bbclass | 8 ++++ 4 files changed, 74 insertions(+) create mode 100644 meta/classes-recipe/packagefeed.bbclass create mode 100644 meta/classes-recipe/packagefeed_deb.bbclass create mode 100644 meta/classes-recipe/packagefeed_ipk.bbclass create mode 100644 meta/classes-recipe/packagefeed_rpm.bbclass diff --git a/meta/classes-recipe/packagefeed.bbclass b/meta/classes-recipe/packagefeed.bbclass new file mode 100644 index 0000000000..de07529464 --- /dev/null +++ b/meta/classes-recipe/packagefeed.bbclass @@ -0,0 +1,50 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +# Class for creating package feeds with indexes for a group of packages and their +# dependencies. +# +# When deploying a package feed or repo for an image such that deployed images can +# use them, it's not always desireable to include all packages in the default +# DEPLOY_DIR_ directory in the feed. This class allows specifying one or +# more packages (or packagegroups) to define what should be included in a feed and +# creates a feed in DEPLOY_DIR_FEED with only those packages and dependencies. +# This allows for creating one or more feeds at once and removes the need to +# remove unwanted packages manually before an index is created. +# +# Example: +# inherit packagefeed +# +# RDEPENDS:${PN} += "\ +# packagegroup-core-buildessential \ +# " +# +# Optional variables: +# FEED_DEPENDS: +# Specifies one or more feeds that the feed depends on and excludes packages in +# those feeds from the feed being built. Used if a feed depends on packages in +# the feeds listed and it's safe to assume those feeds will always be present. + +PKGFEED_INHERITS = "${@' '.join(['packagefeed_' + x for x in d.getVar('PACKAGE_CLASSES').replace("package_", "").split()])}" +inherit ${PKGFEED_INHERITS} nopackages + +LICENSE ?= "MIT" + +# Feeds listed in FEED_DEPENDS and their dependencies will be excluded from the feed. +# This allows for side-by-side feeds without duplicate packages. +FEED_DEPENDS ??= "" + +fakeroot python do_packagefeed() { + from oe.package_manager import generate_feed_dirs, generate_index_files + + generate_feed_dirs(d, d.getVar("FEED_DEPENDS")) + generate_index_files(d, isFeed=True) +} +addtask packagefeed before do_build +do_packagefeed[recrdeptask] += "do_package_qa" +do_packagefeed[nostamp] = "1" +do_packagefeed[rdepends] += "${@' '.join([x + ':do_packagefeed' for x in d.getVar('FEED_DEPENDS').split()])}" +do_packagefeed[cleandirs] += "${DEPLOY_DIR_FEED_DEB}/${PN} ${DEPLOY_DIR_FEED_IPK}/${PN} ${DEPLOY_DIR_FEED_RPM}/${PN}" diff --git a/meta/classes-recipe/packagefeed_deb.bbclass b/meta/classes-recipe/packagefeed_deb.bbclass new file mode 100644 index 0000000000..2decc70a4f --- /dev/null +++ b/meta/classes-recipe/packagefeed_deb.bbclass @@ -0,0 +1,8 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +do_packagefeed[depends] += "apt-native:do_populate_sysroot" +do_packagefeed[recrdeptask] += "do_package_write_deb" diff --git a/meta/classes-recipe/packagefeed_ipk.bbclass b/meta/classes-recipe/packagefeed_ipk.bbclass new file mode 100644 index 0000000000..89d296200b --- /dev/null +++ b/meta/classes-recipe/packagefeed_ipk.bbclass @@ -0,0 +1,8 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +do_packagefeed[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot" +do_packagefeed[recrdeptask] += "do_package_write_ipk" diff --git a/meta/classes-recipe/packagefeed_rpm.bbclass b/meta/classes-recipe/packagefeed_rpm.bbclass new file mode 100644 index 0000000000..8ce37cc855 --- /dev/null +++ b/meta/classes-recipe/packagefeed_rpm.bbclass @@ -0,0 +1,8 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +do_packagefeed[depends] += "createrepo-c-native:do_populate_sysroot" +do_packagefeed[recrdeptask] += "do_package_write_rpm" From patchwork Wed Aug 16 20:08:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B04CC25B73 for ; Wed, 16 Aug 2023 20:10:11 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.132]) by mx.groups.io with SMTP id smtpd.web10.170902.1692216604866387376 for ; Wed, 16 Aug 2023 13:10:05 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=PhXRu+qj; spf=pass (domain: ni.com, ip: 40.107.237.132, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=asTNrYyD8IgqLGcAV/wiugeeI12PTm1sNkgXdFZATQc3Z3/RVue5iAtUelpW2T4+VDqWramkHtFdYbHuYQbBPiKyz+7+69Cy4etHreS4xGb5QewJqb2CKxTVvyEDewmE4Ezf51kshW6ZmF+5uA4esZqb9gjg+hRuMTEQ6+VSbZPAjkTV5VOb8L2CqFEzTAM5HBIRb2ZUNtEgvp624qp850IvXn8vTCXKsYss5vsqFhWdZR/DuV9ZP6TbO+4VbJhZSc/rOUU06MgGlayIN3sdDhRjpnqfuawdhEze2OUHSdHXlrHZ1ip01a0MWxrUDa/3+beCHI1PT9EYSZbqJTpkzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fK7VbTET03N7N+NthNL6A15B+qMG88w9aD9O+4SEUo0=; b=PToquEZ4ZkgQW9t7dineCJ5vJu7ddKN2pypHdJT6yId83qMEnLzsvHwCfwjT3/X5n+mM5lNZpzSTa8pG9rV1y8mSrY0zITO0IvW5MFut8nQkbSqI2/kJBqrNKCxaMI48R340gpF5KLQBcLxRXxNfyHcK3jOQ1CEuN+DSprr/SogZToaHq7R5i0eh07CaAf2T0AtRCe4rMUNn7fEYT3F5JtVBA1cUzFuBbPhnHHdvLzhCKeAWFCRQMEIMB9qW6ZmQiXy90oQ6Zgdnimecj464pDKS0BU4df2UARVHoR2SfFBykoK9L9YeIsIiRhhiAyl6JQye3Dfxu7CF0vi34cpNxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fK7VbTET03N7N+NthNL6A15B+qMG88w9aD9O+4SEUo0=; b=PhXRu+qj4I4DLzqrcO+3p86LvJulvWizyMH4cP7sisDA8ikLabF3NZojIV0CjqD+GsZ5k1xtZseqDpq0V3qcom5MLKoXr1kHk4jBdzNuKNICK/Y4ZebqvZTwMavzI2TlT1JPT5r+I4ezpshfYeo+whNTQg+iknUPK78NMMw3qWw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by BY5PR04MB6963.namprd04.prod.outlook.com (2603:10b6:a03:228::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.29; Wed, 16 Aug 2023 20:10:03 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:10:03 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 6/7] packagefeed.bbclass: Add cleanfunc for cleaning feeds. Date: Wed, 16 Aug 2023 15:08:36 -0500 Message-ID: <20230816200931.2577181-7-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: SA9PR13CA0003.namprd13.prod.outlook.com (2603:10b6:806:21::8) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|BY5PR04MB6963:EE_ X-MS-Office365-Filtering-Correlation-Id: 0d9682cf-a679-4d24-8a84-08db9e94c6b4 x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jzL9aiNoY5ZC/TFnms0qidRHGh3NyoJRuGoOjICzuOenw7U8lZeOMa0NxKiTzxJJ3HQjRZMdfdVZbFUSB3ZL8puoJm6KBBBv6ivvHnhJ+NVaDJHNwLFrwrWWhy7z8Ibuz3ZmgZxWUkPa1jNM5gZsIQXm5btwTHL8Jkt0rRedrHV7HGInGXX1WLRI89COpuqkvdzrMYa4Z4a+/CrXPTcgM9s6dfiVAWop7k4B3HZTFANyXtK4nebnCQGvAq/BieQgd6Cq3LrSKL7N8fUSupof38MRDtdh8RN3Cv/CfAYw2sWyXrdQTv1gAO54GYrDdr0dD3/dRo/p0pX3y1l2cBOJFr8jd2l4ZsfIstYlohaal4P+SAOzyvVb0bIqJTie2jCJ46vyNmilbpIFjzMu4CcU7itzgngNHcp2xsX3EyPVHRXY2wXXSGrxHlmlJAL4y7+4+S7xWrBNFLzqXCY4ImubTUIiaCoWFpNAjGXYLqn3cZ8BDkUZC5iibN3nwOLeC72rXUhDV3JWuZ39Zgm+mVRTCAdMWf76DaeyQT78O6odYPDVi8OWlYB+HzveWkIagoxT X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39850400004)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(83380400001)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: aTX3aqV7Wak1MwYbO6ZiY8lrP/7eowtl2kkzIuB5YTUpx6Ofjuh/gm+CmIv1ZOhzWy74CPkqefU834S0drJ7k6RrlGSppvQfXqVYx/NVjmz0+F5HJQMiZEKU29CzxepZ8Ef0iBhk65r6A1pJZzfl+hBadSqxZ+9fgArW9eFD3d5FtJ5GHGd7fa2M2iDcrxr3FeoQGR/1ZMbIEv4SipTxAQTbngf95eP5aLl7RZdDO3J9AYPrr2rueFO3U5b70O9uG9Jnz5T7zOBdH4eMA65kbWA7j1bZrSJV3gOhdY/C2tD7vqIumsVWz1tzMvYVo14//NlIy4NEWQY1nHwjb07P+uv4Xe2LzSOOPKCoZQQ6rd3PmTSfWTiHir/tHRVBL1ZHbZ1AWjWxx2mWKqGmjE9LKyxRi+fLNhyKrm0JogVIGOr/6wQANxyEMuVAMa2vbZgTSjMAp7QoasVEYN7GcDx3+1PmYqkwLZf+lW3BWY8xm/vrB0tfBREcTjv7ab8HhYRL5jkVq21QsS8sGWQKsnqPaTVsXbKtVzhe64RbIRG8LJbRW2q+NP3AT3mggdsZAXU0c5foLl92u49inPZ4kGHG1NQqQ53cLFxFDzVi+InR2ir4zOKH/87iFhfCY/pvFfu6oKyGuwoRm0Dd9j1KbVPSYHmRTM09D1fS39cDRutj1xW30i9PnhkwH2XWRKI8QkoqjzdS9rSNagY0h7oev+G7KWp7w8QoVEfnDmLX5a0teqByfLv3pdCz5JeNLd6J4HuLjXxUzaDRw1XjD7b78iIiZq+d7A42SJlcooL1/q8EaTKgCY6Fs/xxNg2ardmQ4Ccp+2/XyxFlRc5uj4XIbZyDKO5ph+VtbbX/0Xet0grYGUa/GoJQKzxfehSxjLdeL0Rvm1N8phKxdeIZnR0CjNHq5gUQYgVw8pIJACYbFA6V0tYSW1kXkEEFv8C8mFzXpoSqD+eY33t218JHkXaj4uBV1fDeyiHgrCLrOsCpuEBBZg8EIVqTtNc6K5GeG4oKDY8hJpOjjBNYBDu1G5BQrNnD3Ia/MO8bGKE9Tij0LSo0W0Yen15EvCjspCx2Wc1g9IY/K/KcHQFNtG2378sSz2XMcd/O0tPNmSZO5qntSgC8cemyL69iNMraXxjpniJyATiExmnTI7U1/A8TFfQQOz972J+PhbFCAHjIywwaQzV17WjOQYgctAzw4YnBFrnBlmRogzf7J1g5W9brhxwiHWLl1w56iwqI5d3LvTFBL1xKv5tfFDtwrq6XvKYPFrKD7OxzcCKDUThjo57OX/JdwO+Rb26dVbkjem39jxL/NM+uNH3yuvg2jPUT9G3V9mnb2M9kTfp8DTUKqqLm2G+QvqnsmaXhn4CqT/1tbimmp2JgUvkp3kuUr7AgqipYvj4R1vGrCKJRQ2rW01HTgCzJAAuKTvPa/EEtanZ+cNiWNGvUzhntB5c1MqGgiBiS/+uKIMXogcj+hO1Dn6kEpHmalUd7sbtj70ryg4Njt8VvKEJPB7k7dYZHqW7xK3/xT8jGbWT7Nz6/hDaVAQ7KC8Tz7Rnt/0s4/pNtL9ktd+NS518pzu2khLIf+9ypNeOl0PB9BUBJKqiBA0elUcMHu6NF5fM+Mg== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0d9682cf-a679-4d24-8a84-08db9e94c6b4 X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:10:03.0821 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aZUTxJrngZNth3nUxi3535Ttw+EJ0r5ZJfunoEKSsTZ1rUsYP0EFuCVTPKaq5uBjnEk2dqrbhZ0WaEnGix6UmA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR04MB6963 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:10:11 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186149 Since the packagefeed build logic does not use sstate, the deploy directories will not be cleaned by a do_clean or similar commands. This change adds a function to wipe all feed deploy directories for the given feed when a clean command is run. That is, regardless of the value of PACKAGE_CLASSES, all DEPLOY_DIR_FEED_ directories will be cleaned. Signed-off-by: Charlie Johnston --- meta/classes-recipe/packagefeed.bbclass | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/meta/classes-recipe/packagefeed.bbclass b/meta/classes-recipe/packagefeed.bbclass index de07529464..0bd37790eb 100644 --- a/meta/classes-recipe/packagefeed.bbclass +++ b/meta/classes-recipe/packagefeed.bbclass @@ -47,4 +47,16 @@ addtask packagefeed before do_build do_packagefeed[recrdeptask] += "do_package_qa" do_packagefeed[nostamp] = "1" do_packagefeed[rdepends] += "${@' '.join([x + ':do_packagefeed' for x in d.getVar('FEED_DEPENDS').split()])}" -do_packagefeed[cleandirs] += "${DEPLOY_DIR_FEED_DEB}/${PN} ${DEPLOY_DIR_FEED_IPK}/${PN} ${DEPLOY_DIR_FEED_RPM}/${PN}" + +CLEANFUNCS += "packagefeed_clean" + +python packagefeed_clean() { + bb.note("Cleaning feed directories for %s" % d.getVar('PN')) + + deploy_dirs = [d.getVar("DEPLOY_DIR_FEED_DEB"), + d.getVar("DEPLOY_DIR_FEED_IPK"), + d.getVar("DEPLOY_DIR_FEED_RPM")] + + for dir in deploy_dirs: + oe.path.remove(dir) +} From patchwork Wed Aug 16 20:08:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Johnston X-Patchwork-Id: 28934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13D80C25B76 for ; Wed, 16 Aug 2023 20:10:11 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.103]) by mx.groups.io with SMTP id smtpd.web11.170836.1692216610559862245 for ; Wed, 16 Aug 2023 13:10:10 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="dkim: body hash did not verify" header.i=@ni.com header.s=selector1 header.b=E/Nfdb9I; spf=pass (domain: ni.com, ip: 40.107.237.103, mailfrom: charlie.johnston@ni.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T1FjTuiPtQFxRINYbZuSY5ugSncHaRELWTewMLwsSA8Q5P6AtMJVYHFNhCSRU1gUQ/PhdsgyFkCVYTVRBFOWEvAXN/REFMr+400rs/hD38nVhA1dyj27iciXGoKXz73xXuxyylxz7rwvigBpsJVcVi8flobQQ1gbw3mQI7kJfYfdgva2xMtAF7ztcKBkjJ+WNOACq7jZlvjNtHhVlxvf/1p/GxJzd1TOqVyxmd8anY6UMvNZaUBJ9A2jpo3ogqap20TUT5x3nUDC51XEALJ+sK/Euz2QG3b9syrzZDKNdfgaaQ6a5k5SjeSI2crnDi10O+C7ooOw33fFTaNMtRaC+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wDHHd6ABQPkVGtUwrAAfdu7lxktQ43NJx7kAf+JkDLA=; b=jrLGIlphdOVQqUjzRjpgjPcyhbWxh6NcIKGtHocEDMzbFLrp4MG+64lUINw4MS/2rPcPl9jTZBfUD2DqNXEyZbocwEoSSLiGp5IpF38LZ5N9qlpg/gRdxSzqYBOlVuEWS6a/UWZ5WEoYgPajEtSTILI0myZyD4VABhZTlPygXTz9L3myO70esn64Dw2us5ZKCJFgjSiw/OPIxTwVfSBXLdXJsMbyraj1cYAZYAHwQdwfEZVlM8szZTuOV2ihN0z3X9uTvN57DwrNTvAbIIuGvxr50v38b1VSUFMtfPQi0ObpTVbowH73yK32fRQNunb9oVP829gQl09b6HyXrJeR3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ni.com; dmarc=pass action=none header.from=ni.com; dkim=pass header.d=ni.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ni.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wDHHd6ABQPkVGtUwrAAfdu7lxktQ43NJx7kAf+JkDLA=; b=E/Nfdb9I6jshWk7ClUztBjjqeIafwWkNpg4ee1nirIBEtJ4/GjfU2TdoFab0QF+42M4oXCLtNnrQumnE3nt67d4p3uD84vsqqkslvbr5NCDXInreJvVOZOI4xMxWuJu6drvO89EXCn+EW1gGuYpmVq0zRkSojVBmgDCdUUnIOfg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ni.com; Received: from SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) by BY5PR04MB6963.namprd04.prod.outlook.com (2603:10b6:a03:228::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.29; Wed, 16 Aug 2023 20:10:08 +0000 Received: from SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11]) by SN6PR04MB4879.namprd04.prod.outlook.com ([fe80::c738:ef27:97ab:fa11%7]) with mapi id 15.20.6678.029; Wed, 16 Aug 2023 20:10:06 +0000 From: Charlie Johnston To: openembedded-core@lists.openembedded.org CC: Charlie Johnston Subject: [oe-core][PATCH v2 7/7] testimage.bbclass: Update rpm dnf tests to use do_packagefeed. Date: Wed, 16 Aug 2023 15:08:37 -0500 Message-ID: <20230816200931.2577181-8-charlie.johnston@ni.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230816200931.2577181-1-charlie.johnston@ni.com> References: <20230816200931.2577181-1-charlie.johnston@ni.com> X-ClientProxiedBy: DS7PR03CA0185.namprd03.prod.outlook.com (2603:10b6:5:3b6::10) To SN6PR04MB4879.namprd04.prod.outlook.com (2603:10b6:805:9b::29) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN6PR04MB4879:EE_|BY5PR04MB6963:EE_ X-MS-Office365-Filtering-Correlation-Id: 2f0cc0b3-cc36-473c-2eac-08db9e94c8a9 x-ni-monitor: EOP Exclude NI Domains ETR True X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: t0WUZI2kXRVapvjJcAhJh04PsLu0mT6l3CiIQo/FORJzYdnyP++vASw2+UYzm/XsFJD6y2WMXHQWMMON9CPzaxlxEz8w67edvFnsLUHTqHG61J9/HH4h7BxA2oY99oBwTIHKPJPlYsB//RxxF0GDqsHeFBC6K8VXp5TGcixnMwa802aoC16EAnI4E2qbuAUTwNnDwPDtvhm7CXD4DMOIkorVn4ptUrDbDsrbWiM7b2y5Rv9mKT6rLbbrpFIi91qs12BIEgpr/P6Chaohexn4xPdB/V5MQCiON+ce9nxJuVfBOK2SU14h1LphEoKMzN+W/HwskpSmPT8dgxPBDb69Kkyba8PAA/FPzIgfi6khgwcTVxIIMMZqYZnxNEssmJy3G/spTiL16wF9sPlg4bo1MM8sIu90vC+hEL2+x/x4DEwm0B/+w9VJp/WXpzOULdNLjOUhZbOL821J1owwaGj3hUUWfBM2g4VjAdEWFxirsXH0B07X+XdCbiAkhZpFTfhhsUm+wtGvsTLntGG69BligTyKLQ7lGy3gNg0mkGsewG1IFmWkuTrJHNz/q6gXhzjy X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR04MB4879.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(39850400004)(346002)(136003)(366004)(396003)(451199024)(1800799009)(186009)(316002)(6916009)(66946007)(66556008)(66476007)(5660300002)(41300700001)(44832011)(38100700002)(8676002)(4326008)(8936002)(2906002)(15650500001)(83380400001)(26005)(478600001)(86362001)(6512007)(6506007)(36756003)(6666004)(1076003)(2616005)(6486002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: A1ltvHk3eV1+6Kjez4xmy2+jMgjCPaMyKBKvERsFByewnRO6ps9Lk63vEz33l2d0TylyLUKrs78SPUUHcTSCb9s1H8F/1/mWSL/yaaFwsF89abY9/OQLVMOTOV5NJclryeTfITl49HE/enGCRvxsxrcbKAkPA2E6wtpnoPchlpdeFjVIsK3Z0pmmOvJTO7L87euIBD+2SjM/6oFW9d3oOgcnP1n/jyxGCqeGmah3X2gpfBDKF5HC0zGX7iNkTsuxMZY7hCUfZFC6j8fy5KEteGJZSxuk9je57vfLQyuCTQuGfHLrFWXN5Mpp7u23jGf9HrdP/lU9OIARgdOwrWzBlwEkP0Jcwal3OLsjqbW4iJ6NCkLF2lvpguJbd0ryVNJb6OissDBNt44ZDOAGOi/GbaU3pryQ6dPa0UzMtby/zT3tkuGDw3wefZzxseOz5mms/s7jrYfMML6Cbt0E6Tjye1R0wVXmQlRfKDvvZtNhhUbe+z6z1Sze07DB/YZAQK3lCj708PmRUJu1f2yshgBfCrRWGMA9CxfqeQWvzO5A7oPtoOYFAhJzMA6t5gIDnuxYTvTkzMNUDXRAhx/DRExc3tU/Mhvwfl8mpeYYsffqcsBT7DIq8vqx/QbHR+sLt0XmJUw21pKtn74UILhJzqRngCHalouVLB8BXK7putu+zDPKao/6O4vcX6QVwBbKTzduqrbAzqnT2GOOEHoPwM0gFK3XNprAfu7moN7ytr40nNsImnAegqYj0HQf4ztWxQXP1rn7lZJ+oHLtz1Fz3KZSqokK8lTfx9E5EttRn81JRi4cFuu9wvHyZwoE2QgKAsSZF3qEFOldrEx1W5jjq0ZTyKSEx9LCV7nPYGT6VHbGyVnAZ25RVKes1v0H8Q6FvQLEg9p0+uR68TSCvZPtvYN3undOQ/HVmI6PF6fHYpdRp6XmdKppOWpwk4Slv5kOxrkyHYcZ6uwUVyjbipxkL18Fr8Ra6p5Sg+qDiceCpv3r9R0ZN+LrlYY/ZfnDggjL/2DdGcAMF6K8BmGwcmzwh+mbBc+gwGL6z/E3i+d7LW/J/bTc1UiMkBWTUSNhtTF9cVjCObDMq5kq80qk7TLq/gqg/1u+RhmFgPlmTfADgCbiUCLiS77W/n19W1Z4JBTnsxZD2cs4ZXYcs89A07bJrWUZcOqpVNHcI+wd2YK2f25my6F3xeukDCgjveeDmGJnNQUB6K/fFC1GYhb8/659bFe8jOtfdFp1ASWNKMAjXI+D4mfxXJkEhDVzz0gHQdH3CfWb6vAjdXwm7VItc8optQ8gWDCBqJA3aqwan5HQumhkKwB3JZCq1vaPIG/uYTNCsNPJoSLqkzIcNjML42dnB/Ri1Tk7PgFKkkMLh0x8sZfi8/aFBrUsvP66Esyesrge5W475mJ64XXhE8Xv/cuAOq57kkuGrSD4KFOpQuzqSp/LwcpPhvAFHUbi9qo3IpEdAD/VDpq0f+ZUOVunhSv5hx+jwtRDg+za2r9qrrhSTDmdnq7wK4cgl6qUqA8LqWNg+0nFGUzIz1kS7Js+/Zykp0V19gJCEoeePF5zHnpvj+0tKFXnaQ7qCuKh6jCtNczJkVMrR83XVEPshxBMqw+5o7rwKw== X-OriginatorOrg: ni.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f0cc0b3-cc36-473c-2eac-08db9e94c8a9 X-MS-Exchange-CrossTenant-AuthSource: SN6PR04MB4879.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Aug 2023 20:10:06.3121 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 87ba1f9a-44cd-43a6-b008-6fdb45a5204e X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mZ/VyzSUeo04aCaDjWDFjh3qrRgPH9M1nX+RlVoClNqX8mqCZQMXuNGNfxxd0M/fU4VSBE0q/Gm+tztdgjPU6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR04MB6963 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 16 Aug 2023 20:10:11 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/186150 To test the new packagefeed.bbclass, the testimage case for testing on rpm images has been updated to use a newly added packagefeed (packagefeed-core-rpmtest) instead of manually moving the feeds from DEPLOY_DIR_RPM, removing unwanted packages, and building the indexes manually. The new logic creates the feed in DEPLOY_DIR_FEED_RPM via building the packagefeed and links to it from the location expected for the tests to run. The resulting feed is technically bigger than before but is still smaller than creating a feed with all packages present in DEPLOY_DIR_RPM without the manual removal step. These changes were tested against testimage runs of the following images: - core-image-minimal - core-image-full-cmdline - core-image-sato - core-image-sato-sdk Signed-off-by: Charlie Johnston --- meta/classes-recipe/testimage.bbclass | 57 +++---------------- .../packagefeeds/packagefeed-core-rpmtest.bb | 14 +++++ 2 files changed, 22 insertions(+), 49 deletions(-) create mode 100644 meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass index e3068348ff..d862e6d98e 100644 --- a/meta/classes-recipe/testimage.bbclass +++ b/meta/classes-recipe/testimage.bbclass @@ -101,6 +101,7 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'dnf-native:do TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'createrepo-c-native:do_populate_sysroot', '', d)}" TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'ipk', 'opkg-utils-native:do_populate_sysroot package-index:do_package_index', '', d)}" TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 'apt-native:do_populate_sysroot package-index:do_package_index', '', d)}" +TESTIMAGEDEPENDS += "${@oe.utils.ifelse(d.getVar('IMAGE_PKGTYPE') == 'rpm' and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES')), 'packagefeed-core-rpmtest:do_packagefeed', '')}" TESTIMAGELOCK = "${TMPDIR}/testimage.lock" TESTIMAGELOCK:qemuall = "" @@ -251,7 +252,7 @@ def testimage_main(d): if (d.getVar('IMAGE_PKGTYPE') == 'rpm' and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES'))): - create_rpm_index(d) + copy_rpm_repos(d) logger = make_logger_bitbake_compatible(logging.getLogger("BitBake")) pn = d.getVar("PN") @@ -459,55 +460,13 @@ def get_runtime_paths(d): paths.append(path) return paths -def create_index(arg): - import subprocess +def copy_rpm_repos(d): + rpm_dir = os.path.join(d.getVar('DEPLOY_DIR_FEED'), 'rpm', 'packagefeed-core-rpmtest') + idx_path = os.path.join(d.getVar('WORKDIR'), 'oe-testimage-repo') - index_cmd = arg - try: - bb.note("Executing '%s' ..." % index_cmd) - result = subprocess.check_output(index_cmd, - stderr=subprocess.STDOUT, - shell=True) - result = result.decode('utf-8') - except subprocess.CalledProcessError as e: - return("Index creation command '%s' failed with return code " - '%d:\n%s' % (e.cmd, e.returncode, e.output.decode("utf-8"))) - if result: - bb.note(result) - return None - -def create_rpm_index(d): - import glob - # Index RPMs - rpm_createrepo = bb.utils.which(os.getenv('PATH'), "createrepo_c") - index_cmds = [] - archs = (d.getVar('ALL_MULTILIB_PACKAGE_ARCHS') or '').replace('-', '_') - - for arch in archs.split(): - rpm_dir = os.path.join(d.getVar('DEPLOY_DIR_RPM'), arch) - idx_path = os.path.join(d.getVar('WORKDIR'), 'oe-testimage-repo', arch) - - if not os.path.isdir(rpm_dir): - continue - - lockfilename = os.path.join(d.getVar('DEPLOY_DIR_RPM'), 'rpm.lock') - lf = bb.utils.lockfile(lockfilename, False) - oe.path.copyhardlinktree(rpm_dir, idx_path) - # Full indexes overload a 256MB image so reduce the number of rpms - # in the feed by filtering to specific packages needed by the tests. - package_list = glob.glob(idx_path + "*/*.rpm") - - for pkg in package_list: - if not os.path.basename(pkg).startswith(("dnf-test-", "busybox", "update-alternatives", "libc6", "musl")): - bb.utils.remove(pkg) - - bb.utils.unlockfile(lf) - cmd = '%s --update -q %s' % (rpm_createrepo, idx_path) - - # Create repodata - result = create_index(cmd) - if result: - bb.fatal('%s' % ('\n'.join(result))) + oe.path.copyhardlinktree(rpm_dir, idx_path) + # Test cases assume only arch directories and not a top-level repodata next to them. + oe.path.remove("%s/repodata" % idx_path) def package_extraction(d, test_suites): from oeqa.utils.package_manager import find_packages_to_extract diff --git a/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb b/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb new file mode 100644 index 0000000000..9cb0b933b9 --- /dev/null +++ b/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb @@ -0,0 +1,14 @@ +# +# Package feed containing the packages needed for rpm based testimages. +# + +inherit packagefeed + +RDEPENDS:${PN} += " \ + busybox \ + update-alternatives \ + glibc \ + dnf \ +" + +SKIP_RECIPE[packagefeed-core-rpmtest] ?= "${@bb.utils.contains('PACKAGE_CLASSES', 'package_rpm', '', 'is intended only for tests where PACKAGE_CLASSES includes package_rpm.', d)}"