From patchwork Thu Jan 13 14:37:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B049C433F5 for ; Thu, 13 Jan 2022 14:37:37 +0000 (UTC) Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by mx.groups.io with SMTP id smtpd.web09.9579.1642084656854434164 for ; Thu, 13 Jan 2022 06:37:37 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=pa+vjD85; spf=softfail (domain: sakoman.com, ip: 209.85.210.170, mailfrom: steve@sakoman.com) Received: by mail-pf1-f170.google.com with SMTP id i129so38384pfe.13 for ; Thu, 13 Jan 2022 06:37:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BGiwPqTshkR5TFPXFhWfwTbXXgW1IJ0TdGrNYPbEWg8=; b=pa+vjD85Mvj2XSAowcbst6/YdRX3AlDHDK86i9MQVqKBfDzidtxxh7BVH0mthaFLN5 vfLE4sbn80cyxaG1mpoNmbqHTnrAmtlxhc+eLcJH8VzYULrwJnsLjxEWaXunttfZ66Es 8S6yt8NFYPJbR0GxaosfAOa3UYIaFuVt9ku+XJkUil2QYhoxQe1jHwzJkhOri6sK13zT hXBZb+MrF6swyOK5mGqY0OBtF3dh9yrkIklJ9FuR59XlK2vlW0NjOfBsmiYQkQh9B9EK k5giE7/Dd9GGbYCEd2nScgYdPIyk+S6+ySsB9KmJ2D/L3w7i/JkGi51kZoYLzgm4/lxg 1IMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BGiwPqTshkR5TFPXFhWfwTbXXgW1IJ0TdGrNYPbEWg8=; b=blbFZtOf80q1lXv/dQunkThD3MOZuneJO4M14WkZCg3GM59MN2pZsZ8enMNQJ6yi+u PSmDDDuWnO0i5mUlZIJLt9Y7ofvmfUBxR5U4spjr5hx3ivWD0lmLFZkbNfybaYvA1CqA RppLj6TMICLVbGJbGh53LuY9AqGLZpsns8JDW6/Hd/+9UVjfuj5PR25ZHxL+xNiD5Zn4 DhQxa8VxQDBM7Xv3jN2cIHD/XdAYe8xW+qNJVIP6Hdol/7eaziYWhSmt8Lr52TW45G4e XYZ4cHmvuW/lPW2OxH5Ro/z3xyQkFwNdw7+MC2c0eJ3skaTGLlvGFYPxO1NVli9U97Ju CS3g== X-Gm-Message-State: AOAM531XbokgoaicaqSUho0nKDi/56NTkYFmvttQW5e9lF8TfcUwsYFm RTcmzB628ipFxb/SH8D623ypE/FkzVUiBY88KcU= X-Google-Smtp-Source: ABdhPJyvnOavBHkvGiubV4/kzZYktKwDm1hMDVnsRRUxiEZiyFk/GPnSLQSk16jqycLWwwi5QCxy1Q== X-Received: by 2002:aa7:81cf:0:b0:4c0:6242:c14e with SMTP id c15-20020aa781cf000000b004c06242c14emr4427811pfn.83.1642084655676; Thu, 13 Jan 2022 06:37:35 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:35 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 01/11] valgrind: skip flakey ptest (gdbserver_tests/hginfo) Date: Thu, 13 Jan 2022 04:37:05 -1000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:37 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160512 This ptest was already disabled in master branch with: valgrind: skip broken ptests for glibc 2.34 https://git.openembedded.org/openembedded-core/commit/?id=c177c7f9ef6f90ca49074f003accb8e9a1a645aa Signed-off-by: Steve Sakoman --- meta/recipes-devtools/valgrind/valgrind/remove-for-aarch64 | 1 + meta/recipes-devtools/valgrind/valgrind/remove-for-all | 1 + 2 files changed, 2 insertions(+) diff --git a/meta/recipes-devtools/valgrind/valgrind/remove-for-aarch64 b/meta/recipes-devtools/valgrind/valgrind/remove-for-aarch64 index 93bfd45a4e..afa6a94825 100644 --- a/meta/recipes-devtools/valgrind/valgrind/remove-for-aarch64 +++ b/meta/recipes-devtools/valgrind/valgrind/remove-for-aarch64 @@ -120,6 +120,7 @@ drd/tests/tc19_shadowmem drd/tests/tc21_pthonce drd/tests/tc22_exit_w_lock drd/tests/tc23_bogus_condwait +gdbserver_tests/hginfo helgrind/tests/annotate_rwlock helgrind/tests/annotate_smart_pointer helgrind/tests/bar_bad diff --git a/meta/recipes-devtools/valgrind/valgrind/remove-for-all b/meta/recipes-devtools/valgrind/valgrind/remove-for-all index d6a85c4735..9b8db093df 100644 --- a/meta/recipes-devtools/valgrind/valgrind/remove-for-all +++ b/meta/recipes-devtools/valgrind/valgrind/remove-for-all @@ -1,2 +1,3 @@ drd/tests/bar_bad drd/tests/bar_bad_xml +gdbserver_tests/hginfo From patchwork Thu Jan 13 14:37:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98DD9C433EF for ; Thu, 13 Jan 2022 14:37:39 +0000 (UTC) Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by mx.groups.io with SMTP id smtpd.web10.9338.1642084659047412001 for ; Thu, 13 Jan 2022 06:37:39 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=LU7ZqzlK; spf=softfail (domain: sakoman.com, ip: 209.85.216.45, mailfrom: steve@sakoman.com) Received: by mail-pj1-f45.google.com with SMTP id l16-20020a17090a409000b001b2e9628c9cso11389340pjg.4 for ; Thu, 13 Jan 2022 06:37:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DAqWE+22OqK9+BPG3DkvYZh3Q8pAxYX4dozCaGolQWc=; b=LU7ZqzlKButJI0tclR+yqwWhx9/NLeRp/nLTsVd0Wu3Pu0Ew6jwlA+BQEA7whdz8r/ 6t+TXMZwaZbrDkCEe7/Kv7/wGv4qfDpJ4PipZWpVBI6xVGhX30E4+1EdYML4c+Y4cK3W TQzRIdvByNeeC7dqjtcdSEK7X4gjLwF2PaeGdJ6wZWSWv6mFhcFAF2MfoiQrjrvQ4pL7 rftkeTB5Hx5qIw3UI+L/8tQX4qgLZS3nL3bUYtyOoJYfi4kcZh8M3FlP0l8SCSgM1lOy xcE2lwmMYCDl+XAupmL9fXLazKZBSj2Hxi7p2oL1UaM5Q4iZs3krXBAb3lhk5JVo8hzP Q+Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DAqWE+22OqK9+BPG3DkvYZh3Q8pAxYX4dozCaGolQWc=; b=eBCy/Td0USSAJ0pXhysC72f5yJ15i7RlB2dD9IpmWIRa7qjlYmTNfikVrNQpH5tg1l zNtHMZ+3DOslcN44Iafs8XjjnMZ0QzxFElqILYU9lkoJFFeh1OYDlkevcWofJs3Ja72g goXHxx4ybUzZYjBZpVU5splkhgNcWaWTgx+76+n6YH1wF1z7OZ3slIibNmLWGLObkHeQ 4EEDIWIr3cPq8HM2IyB0a7Gd7u2Qd64i4+9TWv0Q53WZ2BUzpCF1bbJdixISV1K4cwF+ 5qac2gCqR1YXYd8eAeD2e77hELo9enMvydagT+HbS0h3IHn5k43MFu8nDUGwuXBkRquo MBJA== X-Gm-Message-State: AOAM532BGSzcuhOs58Wnp84xtvgmuXHG+Jt8WUNdU0DTKr8I+fRhhoqP z63EBM9J8ryFMvPanHX3/NuQhOzciFmlsr1aFlk= X-Google-Smtp-Source: ABdhPJzMMB/Vwg/SnVvfUhgCpk4sCvijqpApTw/yGOyYCO7nlXF7gQ3fJ3o2IC7dWLhctV0CZCpTzw== X-Received: by 2002:a17:902:a982:b0:149:60fc:9ddd with SMTP id bh2-20020a170902a98200b0014960fc9dddmr4652247plb.110.1642084658006; Thu, 13 Jan 2022 06:37:38 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:37 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 02/11] oeqa/selftest/cases/tinfoil.py: increase timeout 60->120s test_wait_event Date: Thu, 13 Jan 2022 04:37:06 -1000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:39 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160513 The test is timing out on fedora autobuilder workers Signed-off-by: Steve Sakoman --- meta/lib/oeqa/selftest/cases/tinfoil.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/tinfoil.py b/meta/lib/oeqa/selftest/cases/tinfoil.py index 0204537d49..4fcbe594c9 100644 --- a/meta/lib/oeqa/selftest/cases/tinfoil.py +++ b/meta/lib/oeqa/selftest/cases/tinfoil.py @@ -100,9 +100,9 @@ class TinfoilTests(OESelftestTestCase): eventreceived = False commandcomplete = False start = time.time() - # Wait for maximum 60s in total so we'd detect spurious heartbeat events for example + # Wait for maximum 120s in total so we'd detect spurious heartbeat events for example while (not (eventreceived == True and commandcomplete == True) - and (time.time() - start < 60)): + and (time.time() - start < 120)): # if we received both events (on let's say a good day), we are done event = tinfoil.wait_event(1) if event: From patchwork Thu Jan 13 14:37:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B971C433F5 for ; Thu, 13 Jan 2022 14:37:42 +0000 (UTC) Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by mx.groups.io with SMTP id smtpd.web08.9620.1642084661227864789 for ; Thu, 13 Jan 2022 06:37:41 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=2c6uIH+5; spf=softfail (domain: sakoman.com, ip: 209.85.214.174, mailfrom: steve@sakoman.com) Received: by mail-pl1-f174.google.com with SMTP id b3so1164628plc.7 for ; Thu, 13 Jan 2022 06:37:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lyezwQBKaGcUthP7sPgevqJIxlN+mmkCjUpgXpeqiCo=; b=2c6uIH+5v/Y75bQSgPwAt9xTjcB0s1zSPGQ/Mg6l5Z2/cQ1oz5BG4Pl1wmBcAXoer8 eoe3Nq3Pr/tlA2LdFUPo+7LefD0mE1HVuwJ3Uz89iWdnXPrwEt5es2b3Gm8SZfh7RmNT WXGJ8mwnxKgbvxRosAAoO6dh5wSZY49lfuK3Ugia25D4c0S43VU64vbkJ/+PutRmHyFj JtGtQbP63mSXGdaiya6LDZFyiL2k+y3t/JIdROij6yxQME643LYYge/fmukiXhvnQYDw sfkbXe5sXAVNH5hEhFM9UYXt4I+EqwhprtNKNjaqqQn+jbx1VewNwKnBhZSEkgwxKjQZ 0AOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lyezwQBKaGcUthP7sPgevqJIxlN+mmkCjUpgXpeqiCo=; b=chJ5vQux/biBZLWYs/RmNvJwpzMXtx2VjAasuftZzdjp4bGQXqpifxiPoP4pH4phE5 /Kiz82qZMxrDn1gDwipxFPsvPGyITxVMt52nXHqdTR59M3F3c+yO81mVFEKKWbsIsEK2 xIMib5wdx7zLfJXTV4q4A3QgSA8sLeCERnldFGv740pYnJRftcOa+xJMrLYNgLAm3gX2 1hsdOB71hoAeTwSp8wRgYHCuBPATwvab835ZNCRalJjO4+eCVSehi377eaXEyKWNzYFI AYlLto8x/tEyoOBNVlB5DnzKjpH8J35o5eFD5JrfZI6jncH7j8ljX7wSa+Udtf0QKKzs xHjA== X-Gm-Message-State: AOAM532NwZGKcD0MGpvHTZCqqEBXTo41OX7CLYRGGsHOKklcbA/80WEB i5mddI6Zsk8GW2riRfEiGciGtgOh+2E2pTto+Wo= X-Google-Smtp-Source: ABdhPJz17vrVAH2/4Y4Hqb3UwNJ9Fodr8BC4LAnfQn+RelMFAemK7N0fKyys8we6fG7/xYFEJJv5Qw== X-Received: by 2002:a17:902:e80f:b0:14a:43d9:c91 with SMTP id u15-20020a170902e80f00b0014a43d90c91mr4989748plg.160.1642084660273; Thu, 13 Jan 2022 06:37:40 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:39 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 03/11] cve-update-db-native: use fetch task Date: Thu, 13 Jan 2022 04:37:07 -1000 Message-Id: <1ed53d5cfc2be40b2d57b5392ec4d30313209934.1642083419.git.steve@sakoman.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:42 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160514 From: Ross Burton Instead of inventing a new task to fetch the CVE data, use the existing fetch task. Signed-off-by: Ross Burton Signed-off-by: Richard Purdie (cherry picked from commit f5f97d33a1703d75b9fd9760f2c7767081538e00) Signed-off-by: Steve Sakoman --- meta/classes/cve-check.bbclass | 2 +- meta/recipes-core/meta/cve-update-db-native.bb | 9 +++++---- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index b6df2c31da..97cbc63d1d 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -110,7 +110,7 @@ python do_cve_check () { } addtask cve_check before do_build after do_fetch -do_cve_check[depends] = "cve-update-db-native:do_populate_cve_db" +do_cve_check[depends] = "cve-update-db-native:do_fetch" do_cve_check[nostamp] = "1" python cve_check_cleanup () { diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index e86c69803f..50052f8532 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -17,7 +17,7 @@ python () { raise bb.parse.SkipRecipe("Skip recipe when cve-check class is not loaded.") } -python do_populate_cve_db() { +python do_fetch() { """ Update NVD database with json data feed """ @@ -110,7 +110,9 @@ python do_populate_cve_db() { conn.close() } -do_populate_cve_db[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}" +do_fetch[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}" +do_fetch[file-checksums] = "" +do_fetch[vardeps] = "" def initialize_db(c): c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)") @@ -217,7 +219,6 @@ def update_db(c, jsondata): parse_node_and_insert(c, config, cveId) -addtask do_populate_cve_db before do_fetch -do_populate_cve_db[nostamp] = "1" +do_fetch[nostamp] = "1" EXCLUDE_FROM_WORLD = "1" From patchwork Thu Jan 13 14:37:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 889BEC433EF for ; Thu, 13 Jan 2022 14:37:44 +0000 (UTC) Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by mx.groups.io with SMTP id smtpd.web10.9342.1642084663786424808 for ; Thu, 13 Jan 2022 06:37:43 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=prTOQeAU; spf=softfail (domain: sakoman.com, ip: 209.85.214.177, mailfrom: steve@sakoman.com) Received: by mail-pl1-f177.google.com with SMTP id c6so1792609plh.6 for ; Thu, 13 Jan 2022 06:37:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=D0s3LzI2G0y+8p/NWjwmwtm1hMsN00A7WjxeXac0yTI=; b=prTOQeAU2IbZ68Ao5RXUa8qPLA9+fiLJhyKJNamoE8QbCVcCaqGy/MlsbdXeNMpfkZ TbHn9f2vbqj7GLkPboOsIJ1V4iEwHQ5yptmczufgBw/QPyO2i2COD+SwGjXsYWHUCm+z n4FC1OTbIewdYOGF/FpgpTh6lwAPsFfOF1DTFQ/wGtLfcffCxQ7lBRW708qr6ADdptCH v3oCzSDY/CC4Adrel9wDhmG+0nh14X3ZP6Rqf/AsbMa7nLCs9L1fg7OrH/jW/GpDEt4L Rp0a2vptKKo8ImXyaa35Y74V6jwG4MwhKjARqmtuxtPXKn8oj4cB8GSgw9bNvJqaEmm1 N5Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D0s3LzI2G0y+8p/NWjwmwtm1hMsN00A7WjxeXac0yTI=; b=r6ntGgAiSZn8AeNmB44Q1wxpo+RmlMrtxFb6Pmk5WlGSum5LydFp7LqT1/Nz9JFaNh 0LZGSIH73w6aOfba0JaaH4cFeAsguSljeIqxWgb5h+ALvS+MWGePxSL5kbbJLJvFb/Mo rDfjXSUM3O+/iU/ailfem3SnBhqwQrzDyicq6RIxG6SIQaPSVzenSPMBE4H0VZqKYOGq FZJpOmzbIIV7J1tmOmcRRpcVGB5z5WRGN+pDVU4IlFwGFiTXPElFzdTKMVH4Sf9we+iJ mnyvQ0KC9+k+zWP7Y9x107MuKSZzZx1nadSzQ8WZgRnjq7WEkkPFMsJrt8+B8g2yZfSf BqtA== X-Gm-Message-State: AOAM530F5uB5d8jN6s9ss8+h/KdiWeGp4fWlRgYomkS8DLe/WoHVdeoP S0xqSsHII3G6m+NDmBqxZ9XKCpvOJDXLgH43kk8= X-Google-Smtp-Source: ABdhPJwcHBuI2XOFhF6xo+kg8w2cQGY8JZoduZgc3ihCq4dmKa+Vu8reDXGyamYrSvTEv0mttfF0gQ== X-Received: by 2002:a17:902:c40c:b0:149:7d72:15c5 with SMTP id k12-20020a170902c40c00b001497d7215c5mr4698903plk.133.1642084662897; Thu, 13 Jan 2022 06:37:42 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:41 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 04/11] cve-check: add lockfile to task Date: Thu, 13 Jan 2022 04:37:08 -1000 Message-Id: <3c69e4eb08701516150c78cf8b48f3e90d197b2b.1642083419.git.steve@sakoman.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160515 From: Konrad Weihmann this should prevent running into the very rare error sqlite3.OperationalError: attempt to write a readonly database As highlighted by https://www.sqlite.org/faq.html#q5 it is likely that the adapter won't allow use multiple exec calls at the same time. So it's best to prevent multiple accesses at a time, by reusing the already in place CVE_CHECK_DB_FILE_LOCK YOCTO #14110 Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie (cherry picked from commit 677f5741bd265be49d4a5bb933b3e8d8c4eec653) Signed-off-by: Steve Sakoman --- meta/classes/cve-check.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass index 97cbc63d1d..6eecbdbf13 100644 --- a/meta/classes/cve-check.bbclass +++ b/meta/classes/cve-check.bbclass @@ -110,6 +110,7 @@ python do_cve_check () { } addtask cve_check before do_build after do_fetch +do_cve_check[lockfiles] += "${CVE_CHECK_DB_FILE_LOCK}" do_cve_check[depends] = "cve-update-db-native:do_fetch" do_cve_check[nostamp] = "1" From patchwork Thu Jan 13 14:37:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88B13C433EF for ; Thu, 13 Jan 2022 14:37:46 +0000 (UTC) Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by mx.groups.io with SMTP id smtpd.web11.9655.1642084666192339384 for ; Thu, 13 Jan 2022 06:37:46 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=QGVrKEyV; spf=softfail (domain: sakoman.com, ip: 209.85.214.169, mailfrom: steve@sakoman.com) Received: by mail-pl1-f169.google.com with SMTP id t18so10227000plg.9 for ; Thu, 13 Jan 2022 06:37:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=sPsZpm0679vWB7IyH7Kp3GyZaZyUf9RfRXjHOY099oI=; b=QGVrKEyVPBTPEYNkXhP+cMtn6f6QMjDWQC+6m6BYmhzWPfA44X9086KoFiMI6fXR1M n8hG0pw3/0VmtvKsKu6wCFDqc3vRRaS0ECUizoporcZX4wAI6E1e9j/NJLNcwY6cuhmk dS4qpx82R/VJVZ/86HFrmaaAJ1UU2jFAWIABEUBcOY2U1ulnM2qlgggOpqCpqVNUXqDn kC/CjAYUzO1mmEdOAcPgG92DJWDYmrccLcqPIm3/Y2J6IrJa2CulrF8HVrdm8lzClSxm RcDzsXkwoDqH07bEqR3ENeIsM2pnw/tjydgMJiNwaMgJmSQ9IfrQnDWZjSALGVTAGL+L RJMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sPsZpm0679vWB7IyH7Kp3GyZaZyUf9RfRXjHOY099oI=; b=YWCDoM8Fa39r4pGnVCmIYyHxMvItPkWl+kA99rEnrktNcsTfsWroIylKQYlTLS6Xt7 k7BUS4SPWhfVuDt2v2Ith2HTIT6ZCw0uqGHiYmXqrCYgkQq2qvZgLVcbm/GQmGc17luC rxmpz80l/prQqJXcYroOR04HNA9APoCdOdh4D5h1cr2uNhfcQco3BP1l3GVXG9lKae1h bkyuJjt4VCIr/Vc0HAonv19wAFdvNnv+FvN+Pb1D+Z4pxmRjkgM1JAiNy+j/poismgSu /Dpq3vaV+32lFfoTht9fmBHZP5WDQ9YQmxKWBFcgunTw+T6f9tovUrjUBQSkzpanZngh NIdQ== X-Gm-Message-State: AOAM532DZkBGsYhhbziBdYwmktbQTYHxBQaNe7GvSCDwdqZTB6AX4HmR Dfo64N1NGYFiBVurdqA8WVZHNSRVoiKr9oYnKSQ= X-Google-Smtp-Source: ABdhPJyEEHTj1eKW36B9K3TaWPPwLOncl8u28lO9hSVNtBAYw4FezETtXfinZrHCNGx3teVLjeliqg== X-Received: by 2002:a17:90a:bd01:: with SMTP id y1mr14843934pjr.33.1642084665234; Thu, 13 Jan 2022 06:37:45 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:44 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 05/11] xserver-xorg: update CVE_PRODUCT Date: Thu, 13 Jan 2022 04:37:09 -1000 Message-Id: <183a62e7de1ded1f271d8ba4d1b149a85159f4bd.1642083419.git.steve@sakoman.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:46 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160516 From: Anuj Mittal Some of the CVEs have x_server as the product name. Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie (cherry picked from commit 4d5d63cf8605515bb659b6b732683d7fe6540728) Signed-off-by: Steve Sakoman --- meta/recipes-graphics/xorg-xserver/xserver-xorg.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc index b4f0760176..02daafc098 100644 --- a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc +++ b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc @@ -18,7 +18,7 @@ INC_PR = "r8" XORG_PN = "xorg-server" SRC_URI = "${XORG_MIRROR}/individual/xserver/${XORG_PN}-${PV}.tar.bz2" -CVE_PRODUCT = "xorg-server" +CVE_PRODUCT = "xorg-server x_server" S = "${WORKDIR}/${XORG_PN}-${PV}" From patchwork Thu Jan 13 14:37:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D79BC433F5 for ; Thu, 13 Jan 2022 14:37:49 +0000 (UTC) Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by mx.groups.io with SMTP id smtpd.web12.9633.1642084668475312573 for ; Thu, 13 Jan 2022 06:37:48 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=J1+b6J0R; spf=softfail (domain: sakoman.com, ip: 209.85.214.178, mailfrom: steve@sakoman.com) Received: by mail-pl1-f178.google.com with SMTP id e19so10221060plc.10 for ; Thu, 13 Jan 2022 06:37:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1daHS/7DeWHSRA7SFVOGPq87+Wa/gq0Xukrn+Te6S9k=; b=J1+b6J0RW6k45r07+mPVI74D8KuBiz78PI/6bxBpbvdJcfPgahhAJ6jk1hdz6rdCXX kOwGfdMDPRpL/dy3uyulfQqD5rhQnxdgdAnrH7gv0PqxwBo5VYaMSAksk6b+KOvdDa14 CYXo51xuUZjRY+di3S6LTwfi9YkKry6J6bLXgQtv1u4r4hEsv7ZJzfvuZDnUJGQ8PEZ/ k6boe6hlwe37DShGzs3krYjz+OcPYmJtiyBIuxFzvHJ7jvyPDlKLREQYTVlNHsTqBbeX sTPXd3RSZAreWXuMULuIVH2zoo8IaghjXV9Puy0pythz3U3jchfX+9waQVP/FjEjMX2h sXtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1daHS/7DeWHSRA7SFVOGPq87+Wa/gq0Xukrn+Te6S9k=; b=mOvhVTwh2fPRr4Z/ouCu5WOkKpdJCdwpKFwKZ63Mg9gtNtqwLiuLy5EzC1cG3E+S1v +DIcrNs2jEW+ul18I5oveORHwVPRGjPfS14WXwA45APRlm8hNqBgAjtR5pYO03pSuzGM m1qJ3htvIKdUUdAmKECt+oK6Cp820cn1z96zn6xemSR/AyblOhJCWE7sYTZOxyF3OAhn F52/2AU+oPsEak2sNGLx7Zv1x5L+ydapMbcUisaJu/tG7bFgjubzU0UFigdaTyXLcHQk v30UuGsv2aFqLOZf9iyhvkXoZKk3/RGqTDUduZ/FlOyVlr3R/nAmTK+SZDR6eBe2s1j0 rFWw== X-Gm-Message-State: AOAM532XVMNxadFYPVUv4oA3PzLu+fQPtQvqCobTCi3JVdzeOA6Cox38 /2GrDyRW/J+XudnVBshv5EvcpzahP0bVRQT98a4= X-Google-Smtp-Source: ABdhPJzGEOWgifszQrUVzOFEcXNEIFMKLy+yDTiNUT9FydUbXWRbHnbGk16OB4LsvPo3z0+dXuO8Aw== X-Received: by 2002:a17:902:7149:b0:149:c123:a983 with SMTP id u9-20020a170902714900b00149c123a983mr4584607plm.79.1642084667538; Thu, 13 Jan 2022 06:37:47 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:46 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 06/11] wic: misc: Do not find for executables in ASSUME_PROVIDED Date: Thu, 13 Jan 2022 04:37:10 -1000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:49 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160517 From: Ricardo Ribalda Delgado Executables like tar won't be available on the native sysroot, as they are part of the ASSUME_PROVIDED variable. Cc: Paul Barker Signed-off-by: Ricardo Ribalda Delgado Signed-off-by: Richard Purdie (cherry picked from commit 2f574d535f8665b26dab65c14668cf8fc7b751c0) Signed-off-by: Steve Sakoman --- scripts/lib/wic/misc.py | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 8fb508dd39..57c042c503 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -46,7 +46,8 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools", "parted": "parted", "sfdisk": "util-linux", "sgdisk": "gptfdisk", - "syslinux": "syslinux" + "syslinux": "syslinux", + "tar": "tar" } def runtool(cmdln_or_args): @@ -113,6 +114,15 @@ def exec_cmd(cmd_and_args, as_shell=False): """ return _exec_cmd(cmd_and_args, as_shell)[1] +def find_executable(cmd, paths): + recipe = cmd + if recipe in NATIVE_RECIPES: + recipe = NATIVE_RECIPES[recipe] + provided = get_bitbake_var("ASSUME_PROVIDED") + if provided and "%s-native" % recipe in provided: + return True + + return spawn.find_executable(cmd, paths) def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""): """ @@ -141,7 +151,7 @@ def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""): logger.debug("exec_native_cmd: %s", native_cmd_and_args) # If the command isn't in the native sysroot say we failed. - if spawn.find_executable(args[0], native_paths): + if find_executable(args[0], native_paths): ret, out = _exec_cmd(native_cmd_and_args, True) else: ret = 127 From patchwork Thu Jan 13 14:37:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EAA7C433EF for ; Thu, 13 Jan 2022 14:37:51 +0000 (UTC) Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by mx.groups.io with SMTP id smtpd.web08.9623.1642084671159191440 for ; Thu, 13 Jan 2022 06:37:51 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=l/zxXpYE; spf=softfail (domain: sakoman.com, ip: 209.85.215.180, mailfrom: steve@sakoman.com) Received: by mail-pg1-f180.google.com with SMTP id t32so37989pgm.7 for ; Thu, 13 Jan 2022 06:37:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=V8+GkmlnccS/U8LpL5o0s9jLPTKIG63qo4DutjImI6c=; b=l/zxXpYEJe4sB6sQh9rf/7t9GaChmQPN1Lwb8nZGRXvqC6gtNcOliQcERB1tbQTjys D5h2hnYZuVn9gCFdesXuyxxxvScjP2gNstRKmtxfMbYryjAr29fbXhpK9/r+P13SIHa0 LKxR0NQP1BjkywSZ5/H7ciDH6OuF4I+d1AeeMRUWjKickBMB02W6PYT9kKi0TsZQCzUJ 6KPntWkfdclGtd9uhA4QQZhBLlmnhR46Ei+UHmZQi3Vrpi7DU7bPX1ORPfnfW53f5ND1 GBsx94GcaqAu2UbvyTiOyrPcDzpmFCWGlsbbbBRoTnJ6Oc1o3NLBwesAjjh9kr88aRzT 1VnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V8+GkmlnccS/U8LpL5o0s9jLPTKIG63qo4DutjImI6c=; b=iaiq+0tQnVUhYN7zCaUKjC1Py2ITSgiO4esCLe1UVsrHxRTqaP5ayPJ2eUXxE9YAry fAIle1I0D3tcN8FnsEzSt6DJw521g4dFo1dIVA988ulJCt9NxBZ8OPCIXRVLwmWZz1kh PnmwMk30AImS1pV7ZzVX5D5IYkH6HmGjMZDtuRImtb95cFU7xKTYa5WUgDwtwwNqouFg IvcLgbCQCVkkI245gEOELON4On3suMGmmHSHRWxVPPeGa8Ie6z8xl7tWxRzhB3TmH8Di gRsW226Wzwb167HlqyXfWu1Ovt67aLKWJBnOMYIUwdCKaUGZTtAqqSk/2JQXEKUMxy0k IqvQ== X-Gm-Message-State: AOAM530nniR5O1pVzO8cSTnQhAMP5gDEqioKeV/0xEXYMQ/yxe1+YnCI /KCO9N++PefcyNphAZQCiJ3SSCrBdycKeL3vsvI= X-Google-Smtp-Source: ABdhPJz3sR4KAkBRuwWVcTwTwuLlBPNIZP8FwcDLyB+ZxFeaKqjdnkzxRVMUKTxmp+PzKrEqCOvj3w== X-Received: by 2002:a63:7d4e:: with SMTP id m14mr4190332pgn.514.1642084670079; Thu, 13 Jan 2022 06:37:50 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:49 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 07/11] wic: use shutil.which Date: Thu, 13 Jan 2022 04:37:11 -1000 Message-Id: <488815681466d5e4c7640df5281fa6e1f9b4c75e.1642083419.git.steve@sakoman.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:51 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160518 From: Mingli Yu Use shutil.which to find the executable instead to silence the below warning: $ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic [snip] DEBUG: Executing shell function do_image_wic /path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import spawn INFO: Creating image(s)... [snip] [RP: Added conversion for missed function reference] Signed-off-by: Mingli Yu Signed-off-by: Richard Purdie (cherry picked from commit 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87) Signed-off-by: Steve Sakoman --- scripts/lib/wic/engine.py | 6 +++--- scripts/lib/wic/misc.py | 4 ++-- scripts/wic | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py index 9ff4394757..7dbde85696 100644 --- a/scripts/lib/wic/engine.py +++ b/scripts/lib/wic/engine.py @@ -19,10 +19,10 @@ import os import tempfile import json import subprocess +import shutil import re from collections import namedtuple, OrderedDict -from distutils.spawn import find_executable from wic import WicError from wic.filemap import sparse_copy @@ -245,7 +245,7 @@ class Disk: for path in pathlist.split(':'): self.paths = "%s%s:%s" % (native_sysroot, path, self.paths) - self.parted = find_executable("parted", self.paths) + self.parted = shutil.which("parted", path=self.paths) if not self.parted: raise WicError("Can't find executable parted") @@ -283,7 +283,7 @@ class Disk: "resize2fs", "mkswap", "mkdosfs", "debugfs"): aname = "_%s" % name if aname not in self.__dict__: - setattr(self, aname, find_executable(name, self.paths)) + setattr(self, aname, shutil.which(name, path=self.paths)) if aname not in self.__dict__ or self.__dict__[aname] is None: raise WicError("Can't find executable '{}'".format(name)) return self.__dict__[aname] diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 57c042c503..3e11822996 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -16,9 +16,9 @@ import logging import os import re import subprocess +import shutil from collections import defaultdict -from distutils import spawn from wic import WicError @@ -122,7 +122,7 @@ def find_executable(cmd, paths): if provided and "%s-native" % recipe in provided: return True - return spawn.find_executable(cmd, paths) + return shutil.which(cmd, path=paths) def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""): """ diff --git a/scripts/wic b/scripts/wic index a741aed364..6547abe0e9 100755 --- a/scripts/wic +++ b/scripts/wic @@ -22,9 +22,9 @@ import sys import argparse import logging import subprocess +import shutil from collections import namedtuple -from distutils import spawn # External modules scripts_path = os.path.dirname(os.path.realpath(__file__)) @@ -47,7 +47,7 @@ if os.environ.get('SDKTARGETSYSROOT'): break sdkroot = os.path.dirname(sdkroot) -bitbake_exe = spawn.find_executable('bitbake') +bitbake_exe = shutil.which('bitbake') if bitbake_exe: bitbake_path = scriptpath.add_bitbake_lib_path() import bb From patchwork Thu Jan 13 14:37:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 889E5C433EF for ; Thu, 13 Jan 2022 14:37:54 +0000 (UTC) Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by mx.groups.io with SMTP id smtpd.web08.9624.1642084673932389002 for ; Thu, 13 Jan 2022 06:37:54 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=pgVfMoHN; spf=softfail (domain: sakoman.com, ip: 209.85.210.181, mailfrom: steve@sakoman.com) Received: by mail-pf1-f181.google.com with SMTP id a5so39659pfo.5 for ; Thu, 13 Jan 2022 06:37:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ghFLEjoxXIaM6xsUmpag/f+fkPOPH4+x5c4gHQLCq3s=; b=pgVfMoHNXnC5rs8JwdovqYZfpfje7Pw3iXBLgRGlSxQsiE5P05/6wZ4rSnnemkAYGQ ZMty0hVCnL3SMPkJ0fs/CBtdTWS0lpwMocGwVsV70PvQWVX6rhzDMm+MoET9XtKwvBNd uHzT5MLi9cc7veKNlpSO6P2bSSWsImJjkcUok+oN4mu1wcnji8p6k2cdffHyjmK7XTtn e1aaa0Qesr7oIx/7z95VDmmyLnp7BSWl04ooN1CZktdk7L1yxzBH77B94K4DZEjrktJH oSfCfxBNrOEOmfOL46k8r8EE0pjzYYbv7fO18hWp54wMIhJ8pe4wsqKpatEW66MVwf4/ /x5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ghFLEjoxXIaM6xsUmpag/f+fkPOPH4+x5c4gHQLCq3s=; b=qGWvdWzGiAqk0iJvUbn/iTU6xqyShCGd43dLxZXRJUSGH41LgncK7UV+3gAWTHgled 6Dj4pCjjBvQju1jnyqkDJFjrFEPZDj9fiJzaxorvbHa9BupCAG+zG0VDZzLwhpkuFwYL w/qyOoKTg4uibnqYV0nI2gtbSLPUNCGFoCo3UFrDgZfsQ0tKmMCvEUlXI3FPQEC8iRyB U3EPWvRjInWxxAJEYaQgUM+0hWlYR2lU9BJOjCGEATP6TZSSyCJlVbGdmYjJMq4Czdhm FQEOt2YNhphNtuDLD/UT3ukZ3iCs5+R5w89IUG8VTPvKFihJ4s85U3ZQSt0cOCsZqh0Y Mhzg== X-Gm-Message-State: AOAM533o1oN3+klGzFiWtDBhazWVJDgA8dkpFtRgSG0GwZZaQdiVTug3 e4BsykOD0//EwzGgTvI9Q6JXiYg1BHHBa8hAIAM= X-Google-Smtp-Source: ABdhPJyZgJctG/b/3CRVu6F2Q43oup/BN2zp6Ku8Mq74WwqjmALgnntXFtihLmWQkJzv9hUn3SsQGw== X-Received: by 2002:a63:3d8a:: with SMTP id k132mr4150045pga.577.1642084672736; Thu, 13 Jan 2022 06:37:52 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:51 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 08/11] expat: Update HOMEPAGE to current url Date: Thu, 13 Jan 2022 04:37:12 -1000 Message-Id: <022750aaa128189f23063b741bf8396a527713d7.1642083419.git.steve@sakoman.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:54 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160519 From: Richard Purdie Upstream pointed out we were using an old url for HOMEPAGE. Update it to the current url. Signed-off-by: Richard Purdie (cherry picked from commit f3a7e2ba247efe72154c263d1d680aaf3da5b609) Signed-off-by: Steve Sakoman --- meta/recipes-core/expat/expat_2.2.9.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-core/expat/expat_2.2.9.bb b/meta/recipes-core/expat/expat_2.2.9.bb index a2df7ac492..4b63ec89db 100644 --- a/meta/recipes-core/expat/expat_2.2.9.bb +++ b/meta/recipes-core/expat/expat_2.2.9.bb @@ -1,6 +1,6 @@ SUMMARY = "A stream-oriented XML parser library" DESCRIPTION = "Expat is an XML parser library written in C. It is a stream-oriented parser in which an application registers handlers for things the parser might find in the XML document (like start tags)" -HOMEPAGE = "http://expat.sourceforge.net/" +HOMEPAGE = "https://github.com/libexpat/libexpat" SECTION = "libs" LICENSE = "MIT" From patchwork Thu Jan 13 14:37:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88ED6C433EF for ; Thu, 13 Jan 2022 14:37:57 +0000 (UTC) Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by mx.groups.io with SMTP id smtpd.web12.9636.1642084676405928440 for ; Thu, 13 Jan 2022 06:37:56 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=r/GrfamS; spf=softfail (domain: sakoman.com, ip: 209.85.214.177, mailfrom: steve@sakoman.com) Received: by mail-pl1-f177.google.com with SMTP id i6so10363054pla.0 for ; Thu, 13 Jan 2022 06:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=qU/PIAmjla0Ou+9OfNEiZOI+/5xPXiYOl6iCfatP1Tc=; b=r/GrfamSB2PfUaNlp+jEoplHdfpHv8KXaMQwjlgxAUz3Maw0oy6kukf8GEA7PXlDp7 kL2eHGyU6yDu2OyZF8E17GLi4qbcS1EJKxJawP18CWRCggCL6Dt0jkncYx/m1unMCwjD HgpLjg01FaEl34XxAqoaasPuCYzgU/+zPUh/F45pjJU+vO0vX4SYnsY0wa9PDQFlpQzR Bj02ZVNFhWXWwXeMNVaEz6jweuOu9vy5VHfwWG8quAkUS5DWmib744qwJxjOGJbPTOUM Jc0sKw6Uti939h3OrMmyZTwv/rxvMkiI5MHpBoi+FSMMdv4BZz1SnvtW04wIhYui8MP1 ap8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qU/PIAmjla0Ou+9OfNEiZOI+/5xPXiYOl6iCfatP1Tc=; b=ClSa8ePR40nvZOITau6jdwyJoEVGi+Jj/WRJorG43vBA7mgBdtGRRyNT5OXciPhAF+ 4Cc9QlJGqogyzV/fQqd16qxFGgc7xQaqhGJSfZAwTf92JTUjOZwbWbk8fCOO8y6ivZY3 HQYNdd4jx2qUTrKqc9bqWajDmGHhqt7n+Vimws0eAijZao1xW3YdHDFU2eOvnRs/ho8s BZs2FCQK7MQg3whnerzIlKcyfypONUL2cd/Bcd/URnxz2XdnBJ0fNSwORkFIkmWhMxrR B8F4b2R23mMtwrpW3kEgGCC+XNvdZmfs0R78A1FCm0leqvXYXpgmzi+0i08JWWwGHE4F ARJw== X-Gm-Message-State: AOAM533U98k9pu4zqUcfXdtcL1/5FklSyXuZ/3RSxAiEiwUtPRTzj0HR VF7SGFZ0QPmlY/91SDtkkB03W9Fy7Osg/3VAGK8= X-Google-Smtp-Source: ABdhPJxgV5apGrGRidPkRsqg9C/J40YlxdArpqcvtC/a4dFujnshO1u5F8VO6gOxwrFeQZ6TlDDNNw== X-Received: by 2002:a17:90a:d904:: with SMTP id c4mr14669388pjv.77.1642084675390; Thu, 13 Jan 2022 06:37:55 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:54 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 09/11] xserver-xorg: whitelist two CVEs Date: Thu, 13 Jan 2022 04:37:13 -1000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:57 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160520 From: Ross Burton CVE-2011-4613 is specific to Debian/Ubuntu. CVE-2020-25697 is a non-trivial attack that may not actually be feasible considering the default behaviour for clients is to exit if the connection is lost. Signed-off-by: Ross Burton Signed-off-by: Richard Purdie (cherry picked from commit afa2e6c31a79f75ff4113d53f618bbb349cd6c17) Signed-off-by: Steve Sakoman --- meta/recipes-graphics/xorg-xserver/xserver-xorg.inc | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc index 02daafc098..c891211c40 100644 --- a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc +++ b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc @@ -19,6 +19,14 @@ XORG_PN = "xorg-server" SRC_URI = "${XORG_MIRROR}/individual/xserver/${XORG_PN}-${PV}.tar.bz2" CVE_PRODUCT = "xorg-server x_server" +# This is specific to Debian's xserver-wrapper.c +CVE_CHECK_WHITELIST += "CVE-2011-4613" +# As per upstream, exploiting this flaw is non-trivial and it requires exact +# timing on the behalf of the attacker. Many graphical applications exit if their +# connection to the X server is lost, so a typical desktop session is either +# impossible or difficult to exploit. There is currently no upstream patch +# available for this flaw. +CVE_CHECK_WHITELIST += "CVE-2020-25697" S = "${WORKDIR}/${XORG_PN}-${PV}" From patchwork Thu Jan 13 14:37:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89DCFC433FE for ; Thu, 13 Jan 2022 14:37:59 +0000 (UTC) Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by mx.groups.io with SMTP id smtpd.web08.9626.1642084678515125120 for ; Thu, 13 Jan 2022 06:37:58 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=HhkzXosL; spf=softfail (domain: sakoman.com, ip: 209.85.214.174, mailfrom: steve@sakoman.com) Received: by mail-pl1-f174.google.com with SMTP id u11so5077652plh.13 for ; Thu, 13 Jan 2022 06:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1wg2ICW+vfHuk3dqQEu/WLCyGgpvemD4fX+/sXSR1IM=; b=HhkzXosLkeKxg6vntasiSgzj3AFd1XvKCAO4ca5roSwpYmUAYkpnf7dlDqWDCtVlid LUp68C7L0Bd9R2340hhhD0WQ6YmZNSYEFqR5/IAfxhqlLjM2I5k48dLGFjgGHZuOslvI oj69uAGQSn6AaJmr/lJbRSuUnDuOKXuQpBhKuQ+d4HvJI+NbivK+Wv+f00gwOjVlbQnq 06k2JhCNdNeA6Wr7sSCOvq//YdcOdgm7B0j75hPArLppk421/fXTjGVSCdXYpWfTGO0m bEamCpTItFVxA/ThbClE+hSD5QxnkF2Q0xhvt6myCGlviz3XtrdnYQlTQpluVX7Uo38j JT8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1wg2ICW+vfHuk3dqQEu/WLCyGgpvemD4fX+/sXSR1IM=; b=Ia7K/ny87y3fWgjy4osbJznjybx8To+EkEfQt3YxHKuwutLI/PBh9eLQ/7ZFYSR53/ jfWHNQO4nQVvWaZIFDBzbp01wdRriN7pWSl+rbnH4a7e3MDvnqa9jqUeUGdxQxjyTZ7h pqCp9eaUwVT5r+MGpzkTYxodTpPxweeL0cvdslIyDHMZy4lM8ohLzM1NvObBM2uNMBIJ KtB5lTVFK3DfEWSTvsNwD/HINHYWOVMeuUGaUjATZYEdPlJwAtSgYaui6nInGabd/AGR YhsNNrusAEoqmy9Xa//AMaFrEFbxPL6JPoUQvlBjSm93iYCLZXIWlx9/l63GgbXKwrKV vf0Q== X-Gm-Message-State: AOAM531otT92g+oC7JtV17thvwkBLU4wh1SoFGeI3Y3BOH4l1C3lws/W WpgPWXZ4dDWbW+O+4hjuHRi+IFhPFiW9RVEpt3k= X-Google-Smtp-Source: ABdhPJwS/4TZWoWOS1OL3BgPJa1P8ioX8NlRwqzqyOzZB3JMYdJshOjqcJu0tA8G50yhDN8r8E4QnQ== X-Received: by 2002:a17:90b:224c:: with SMTP id hk12mr5421157pjb.62.1642084677615; Thu, 13 Jan 2022 06:37:57 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.37.56 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:37:57 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 10/11] parselogs: add a couple systemd false positives Date: Thu, 13 Jan 2022 04:37:14 -1000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:37:59 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160521 From: Alexander Kanavin New systemd has changed the phrasing when skipping things, with unfortunate use of 'failed': [ 1.623667] systemd[1]: Journal Audit Socket was skipped because of a failed condition check (ConditionSecurity=audit). [ 1.688258] systemd[1]: Load Kernel Modules was skipped because all trigger condition checks failed. Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie (cherry picked from commit 743d09665a4ef743b1fa9ac382a713556dfce1a1) Signed-off-by: Steve Sakoman --- meta/lib/oeqa/runtime/cases/parselogs.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/lib/oeqa/runtime/cases/parselogs.py b/meta/lib/oeqa/runtime/cases/parselogs.py index cc40ee1ac0..1cac59725d 100644 --- a/meta/lib/oeqa/runtime/cases/parselogs.py +++ b/meta/lib/oeqa/runtime/cases/parselogs.py @@ -61,6 +61,8 @@ common_errors = [ "[rdrand]: Initialization Failed", "[pulseaudio] authkey.c: Failed to open cookie file", "[pulseaudio] authkey.c: Failed to load authentication key", + "was skipped because of a failed condition check", + "was skipped because all trigger condition checks failed", ] video_related = [ From patchwork Thu Jan 13 14:37:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 2389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98932C433EF for ; Thu, 13 Jan 2022 14:38:05 +0000 (UTC) Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by mx.groups.io with SMTP id smtpd.web10.9352.1642084684587847487 for ; Thu, 13 Jan 2022 06:38:04 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20210112.gappssmtp.com header.s=20210112 header.b=XhAktJYM; spf=softfail (domain: sakoman.com, ip: 209.85.214.172, mailfrom: steve@sakoman.com) Received: by mail-pl1-f172.google.com with SMTP id p14so10298772plf.3 for ; Thu, 13 Jan 2022 06:38:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20210112.gappssmtp.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=rav351wV+MCYPc3EeEabyDyuj4PXcO9hV9QW5eM/1Dg=; b=XhAktJYMUvhe3XD2Tn3hr0uOZ+D4K0cCek1Rup9qA5UmF5MeUZbavVOJjp0UMrHzs8 pLQDlvbIK/V3KsBaqNCMh9AkK+ak6Gaz1M+3exCjoGN7DVpt7m79RY8RJyvZ7p1kfkoL OuX+9IDcRnJVGdQFmrj1XPZupYI4FI2Krsa/n1Wt1CG6BKGjFrvXFcqWuCjTHKCB/ITi vcHP5h0+OwMFWZSUsbVoM42GK4VYhBeevbSoQnHpaEB0zlAW3zRqEpJfQK2vSbveyqjj CiKJJUWADVoYgC+EBPUAmnsD8p2SkQnigScqh9x8jvnEIQwsEsRiH93wKdtd+Tn13cPM ku9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rav351wV+MCYPc3EeEabyDyuj4PXcO9hV9QW5eM/1Dg=; b=tKQKFwrMZId1G+IjKjcxS9+NIf5xfT8lZ8Jn4lTkLfQtU2+6vZa+gVAfq+mokEFxsk 76plWtH/Ptoi7oMECjcL2a7XaETFiUqA9o5hR407T4v8GAgpyTKPbgSOfqsJPe7utqTJ TizJWSz3qT8XUTa79LWXGYCVXBvj3DYEaEnaM6/clKfEJB2Vbg1bz6FdE1vdmNoPLHIE yLTneA4wE3FHhTv8YHOJgrEVkbDO0m4MBu5oEqpk51WDGpwFNHvZZ7wTRwp/JgEcMe/d N24M1Kvxu1YwY5Mw3PWZx5PtDiBeyAaJgjHe2yljOzDsw2zdoyjIyw0KDJsl3JmFWoPy eLBw== X-Gm-Message-State: AOAM5306bBAHmk0zjDkBNzyr3nsHBu38lNMp4x/WXjO6iNOJYJ03Hrd8 QIkV+ZItQ4qelc7jIeDsIoWexhPTROG29I6JYDk= X-Google-Smtp-Source: ABdhPJwvL9NY3lmunfQ7LmgCaw9+8/c4L7eIVFUDdM52ZI8gRr4QBdhG3Ih79+s+YFIhMteyeNJ+XQ== X-Received: by 2002:a17:902:bd44:b0:14a:2c66:a06c with SMTP id b4-20020a170902bd4400b0014a2c66a06cmr5131530plx.152.1642084682112; Thu, 13 Jan 2022 06:38:02 -0800 (PST) Received: from hexa.router0800d9.com (rrcs-66-91-142-162.west.biz.rr.com. [66.91.142.162]) by smtp.gmail.com with ESMTPSA id g11sm2541544pgn.26.2022.01.13.06.38.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jan 2022 06:38:01 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 11/11] glibc: Add fix for data races in pthread_create and TLS access Date: Thu, 13 Jan 2022 04:37:15 -1000 Message-Id: <01f256bc72fb45c80b6a6c77506bc4c375965a3a.1642083419.git.steve@sakoman.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 13 Jan 2022 14:38:05 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/160522 From: Akash Hadke Inconsistency detected by ld.so: dl-tls.c: 493: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= _rtld_local._dl_tls_generation' failed! caused by dlopen (in _dl_add_to_slotinfo and in dl_open_worker) doing listp->slotinfo[idx].gen = GL(dl_tls_generation) + 1; //... if (any_tls && __builtin_expect (++GL(dl_tls_generation) == 0, 0)) while pthread_create (in _dl_allocate_tls_init) concurrently doing assert (listp->slotinfo[cnt].gen <= GL(dl_tls_generation)); Backported below patch that can fix the following bugs with a lock that prevents DTV setup running concurrently with dlopen or dlclose. Bug 19329: https://sourceware.org/bugzilla/show_bug.cgi?id=19329 Bug 27111: https://sourceware.org/bugzilla/show_bug.cgi?id=27111 Patch: 0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=1387ad6225c2222f027790e3f460e31aa5dd2c54 It requires a supporting patch 0030-elf-Refactor_dl_update-slotinfo-to-avoid-use-after-free.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=c0669ae1a629e16b536bf11cdd0865e0dbcf4bee After adding the above fix there is a number of racy read accesses to globals that will be changed to relaxed MO atomics in follow-up patch given below. This fixes the regressions and avoids cluttering the main part of the fix. 0032-elf-Use-relaxed-atomics-for-racy-accesses-BZ-19329.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=f4f8f4d4e0f92488431b268c8cd9555730b9afe9 Backported the below patch to add the test to check the added fix. 0033-elf-Add-test-case-for-BZ-19329.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=9d0e30329c23b5ad736fda3f174208c25970dbce Previously modids were never resused for a different module, but after dlopen failure all gaps are reused not just the ones caused by the unfinished dlopened. The code has to handle reused modids already which seems to work, however the data races at thread creation and tls access (see bug 19329 and bug 27111) may be more severe if slots are reused. Fixing the races are not simpler if reuse is disallowed and reuse has other benefits so upstream added fix https://sourceware.org/git/?p=glibc.git;a=commit;h=572bd547d57a39b6cf0ea072545dc4048921f4c3 for the following bug. Bug 27135: https://sourceware.org/bugzilla/show_bug.cgi?id=27135 But in glibc upstream the commit 572bd547d57a was reverted as the issue with 572bd547d57a patch was the DTV entry only updated on dl_open_worker() with the update_tls_slotinfo() call after all dependencies are being processed by _dl_map_object_deps(). However _dl_map_object_deps() itself might call _dl_next_tls_modid(), and since the _dl_tls_dtv_slotinfo_list::map was not yet set the entry can be wrongly reused. So added below patch to fix Bug 27135. 0034-elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=ba33937be210da5d07f7f01709323743f66011ce Not all TLS access related data races got fixed by adding 0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch, there are additional races at lazy tlsdesc relocations. Bug 27137: https://sourceware.org/bugzilla/show_bug.cgi?id=27137 Backported below patches to fix this issue. 0035-x86_64-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=8f7e09f4dbdb5c815a18b8285fbc5d5d7bc17d86 0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=ddcacd91cc10ff92d6201eda87047d029c14158d The fix 0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch for bug 19329 caused a regression such that pthread_create can deadlock when concurrent ctors from dlopen are waiting for it to finish. Bug 28357: https://sourceware.org/bugzilla/show_bug.cgi?id=28357 Backported below patch to fix this issue. 0037-Avoid-deadlock-between-pthread_create-and-ctors.patch Link: https://sourceware.org/git/?p=glibc.git;a=patch;h=024a7640ab9ecea80e527f4e4d7f7a1868e952c5 Signed-off-by: Akash Hadke Signed-off-by: Akash Hadke Signed-off-by: Steve Sakoman --- ...ate-slotinfo-to-avoid-use-after-free.patch | 66 +++++ ...hread_create-and-TLS-access-BZ-19329.patch | 191 ++++++++++++ ...d-atomics-for-racy-accesses-BZ-19329.patch | 206 +++++++++++++ .../0033-elf-Add-test-case-for-BZ-19329.patch | 144 +++++++++ ...elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch | 180 ++++++++++++ ...-lazy-relocation-of-tlsdesc-BZ-27137.patch | 56 ++++ ...-lazy-relocation-of-tlsdesc-BZ-27137.patch | 124 ++++++++ ...ock-between-pthread_create-and-ctors.patch | 276 ++++++++++++++++++ meta/recipes-core/glibc/glibc_2.31.bb | 8 + 9 files changed, 1251 insertions(+) create mode 100644 meta/recipes-core/glibc/glibc/0030-elf-Refactor_dl_update-slotinfo-to-avoid-use-after-free.patch create mode 100644 meta/recipes-core/glibc/glibc/0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch create mode 100644 meta/recipes-core/glibc/glibc/0032-elf-Use-relaxed-atomics-for-racy-accesses-BZ-19329.patch create mode 100644 meta/recipes-core/glibc/glibc/0033-elf-Add-test-case-for-BZ-19329.patch create mode 100644 meta/recipes-core/glibc/glibc/0034-elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch create mode 100644 meta/recipes-core/glibc/glibc/0035-x86_64-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch create mode 100644 meta/recipes-core/glibc/glibc/0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch create mode 100644 meta/recipes-core/glibc/glibc/0037-Avoid-deadlock-between-pthread_create-and-ctors.patch diff --git a/meta/recipes-core/glibc/glibc/0030-elf-Refactor_dl_update-slotinfo-to-avoid-use-after-free.patch b/meta/recipes-core/glibc/glibc/0030-elf-Refactor_dl_update-slotinfo-to-avoid-use-after-free.patch new file mode 100644 index 0000000000..dba491f4dc --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0030-elf-Refactor_dl_update-slotinfo-to-avoid-use-after-free.patch @@ -0,0 +1,66 @@ +From c0669ae1a629e16b536bf11cdd0865e0dbcf4bee Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Wed, 30 Dec 2020 21:52:38 +0000 +Subject: [PATCH] elf: Refactor _dl_update_slotinfo to avoid use after free + +map is not valid to access here because it can be freed by a concurrent +dlclose: during tls access (via __tls_get_addr) _dl_update_slotinfo is +called without holding dlopen locks. So don't check the modid of map. + +The map == 0 and map != 0 code paths can be shared (avoiding the dtv +resize in case of map == 0 is just an optimization: larger dtv than +necessary would be fine too). + +Reviewed-by: Adhemerval Zanella +--- + elf/dl-tls.c | 21 +++++---------------- + 1 file changed, 5 insertions(+), 16 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=c0669ae1a629e16b536bf11cdd0865e0dbcf4bee] +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/elf/dl-tls.c b/elf/dl-tls.c +index 24d00c14ef..f8b32b3ecb 100644 +--- a/elf/dl-tls.c ++++ b/elf/dl-tls.c +@@ -743,6 +743,8 @@ _dl_update_slotinfo (unsigned long int req_modid) + { + for (size_t cnt = total == 0 ? 1 : 0; cnt < listp->len; ++cnt) + { ++ size_t modid = total + cnt; ++ + size_t gen = listp->slotinfo[cnt].gen; + + if (gen > new_gen) +@@ -758,25 +760,12 @@ _dl_update_slotinfo (unsigned long int req_modid) + + /* If there is no map this means the entry is empty. */ + struct link_map *map = listp->slotinfo[cnt].map; +- if (map == NULL) +- { +- if (dtv[-1].counter >= total + cnt) +- { +- /* If this modid was used at some point the memory +- might still be allocated. */ +- free (dtv[total + cnt].pointer.to_free); +- dtv[total + cnt].pointer.val = TLS_DTV_UNALLOCATED; +- dtv[total + cnt].pointer.to_free = NULL; +- } +- +- continue; +- } +- + /* Check whether the current dtv array is large enough. */ +- size_t modid = map->l_tls_modid; +- assert (total + cnt == modid); + if (dtv[-1].counter < modid) + { ++ if (map == NULL) ++ continue; ++ + /* Resize the dtv. */ + dtv = _dl_resize_dtv (dtv); + +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch b/meta/recipes-core/glibc/glibc/0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch new file mode 100644 index 0000000000..25beee1d50 --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch @@ -0,0 +1,191 @@ +From 1387ad6225c2222f027790e3f460e31aa5dd2c54 Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Wed, 30 Dec 2020 19:19:37 +0000 +Subject: [PATCH] elf: Fix data races in pthread_create and TLS access [BZ + #19329] + +DTV setup at thread creation (_dl_allocate_tls_init) is changed +to take the dlopen lock, GL(dl_load_lock). Avoiding data races +here without locks would require design changes: the map that is +accessed for static TLS initialization here may be concurrently +freed by dlclose. That use after free may be solved by only +locking around static TLS setup or by ensuring dlclose does not +free modules with static TLS, however currently every link map +with TLS has to be accessed at least to see if it needs static +TLS. And even if that's solved, still a lot of atomics would be +needed to synchronize DTV related globals without a lock. So fix +both bug 19329 and bug 27111 with a lock that prevents DTV setup +running concurrently with dlopen or dlclose. + +_dl_update_slotinfo at TLS access still does not use any locks +so CONCURRENCY NOTES are added to explain the synchronization. +The early exit from the slotinfo walk when max_modid is reached +is not strictly necessary, but does not hurt either. + +An incorrect acquire load was removed from _dl_resize_dtv: it +did not synchronize with any release store or fence and +synchronization is now handled separately at thread creation +and TLS access time. + +There are still a number of racy read accesses to globals that +will be changed to relaxed MO atomics in a followup patch. This +should not introduce regressions compared to existing behaviour +and avoid cluttering the main part of the fix. + +Not all TLS access related data races got fixed here: there are +additional races at lazy tlsdesc relocations see bug 27137. + +Reviewed-by: Adhemerval Zanella +--- + elf/dl-tls.c | 63 +++++++++++++++++++++++++++++++++++++++------------- + 1 file changed, 47 insertions(+), 16 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=1387ad6225c2222f027790e3f460e31aa5dd2c54] +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/elf/dl-tls.c b/elf/dl-tls.c +index 6baff0c1ea..94f3cdbae0 100644 +--- a/elf/dl-tls.c ++++ b/elf/dl-tls.c +@@ -475,14 +475,11 @@ extern dtv_t _dl_static_dtv[]; + #endif + + static dtv_t * +-_dl_resize_dtv (dtv_t *dtv) ++_dl_resize_dtv (dtv_t *dtv, size_t max_modid) + { + /* Resize the dtv. */ + dtv_t *newp; +- /* Load GL(dl_tls_max_dtv_idx) atomically since it may be written to by +- other threads concurrently. */ +- size_t newsize +- = atomic_load_acquire (&GL(dl_tls_max_dtv_idx)) + DTV_SURPLUS; ++ size_t newsize = max_modid + DTV_SURPLUS; + size_t oldsize = dtv[-1].counter; + + if (dtv == GL(dl_initial_dtv)) +@@ -528,11 +525,14 @@ _dl_allocate_tls_init (void *result) + size_t total = 0; + size_t maxgen = 0; + ++ /* Protects global dynamic TLS related state. */ ++ __rtld_lock_lock_recursive (GL(dl_load_lock)); ++ + /* Check if the current dtv is big enough. */ + if (dtv[-1].counter < GL(dl_tls_max_dtv_idx)) + { + /* Resize the dtv. */ +- dtv = _dl_resize_dtv (dtv); ++ dtv = _dl_resize_dtv (dtv, GL(dl_tls_max_dtv_idx)); + + /* Install this new dtv in the thread data structures. */ + INSTALL_DTV (result, &dtv[-1]); +@@ -600,6 +600,7 @@ _dl_allocate_tls_init (void *result) + listp = listp->next; + assert (listp != NULL); + } ++ __rtld_lock_unlock_recursive (GL(dl_load_lock)); + + /* The DTV version is up-to-date now. */ + dtv[0].counter = maxgen; +@@ -734,12 +735,29 @@ _dl_update_slotinfo (unsigned long int req_modid) + + if (dtv[0].counter < listp->slotinfo[idx].gen) + { +- /* The generation counter for the slot is higher than what the +- current dtv implements. We have to update the whole dtv but +- only those entries with a generation counter <= the one for +- the entry we need. */ ++ /* CONCURRENCY NOTES: ++ ++ Here the dtv needs to be updated to new_gen generation count. ++ ++ This code may be called during TLS access when GL(dl_load_lock) ++ is not held. In that case the user code has to synchronize with ++ dlopen and dlclose calls of relevant modules. A module m is ++ relevant if the generation of m <= new_gen and dlclose of m is ++ synchronized: a memory access here happens after the dlopen and ++ before the dlclose of relevant modules. The dtv entries for ++ relevant modules need to be updated, other entries can be ++ arbitrary. ++ ++ This e.g. means that the first part of the slotinfo list can be ++ accessed race free, but the tail may be concurrently extended. ++ Similarly relevant slotinfo entries can be read race free, but ++ other entries are racy. However updating a non-relevant dtv ++ entry does not affect correctness. For a relevant module m, ++ max_modid >= modid of m. */ + size_t new_gen = listp->slotinfo[idx].gen; + size_t total = 0; ++ size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); ++ assert (max_modid >= req_modid); + + /* We have to look through the entire dtv slotinfo list. */ + listp = GL(dl_tls_dtv_slotinfo_list); +@@ -749,12 +767,14 @@ _dl_update_slotinfo (unsigned long int req_modid) + { + size_t modid = total + cnt; + ++ /* Later entries are not relevant. */ ++ if (modid > max_modid) ++ break; ++ + size_t gen = listp->slotinfo[cnt].gen; + + if (gen > new_gen) +- /* This is a slot for a generation younger than the +- one we are handling now. It might be incompletely +- set up so ignore it. */ ++ /* Not relevant. */ + continue; + + /* If the entry is older than the current dtv layout we +@@ -771,7 +791,7 @@ _dl_update_slotinfo (unsigned long int req_modid) + continue; + + /* Resize the dtv. */ +- dtv = _dl_resize_dtv (dtv); ++ dtv = _dl_resize_dtv (dtv, max_modid); + + assert (modid <= dtv[-1].counter); + +@@ -793,8 +813,17 @@ _dl_update_slotinfo (unsigned long int req_modid) + } + + total += listp->len; ++ if (total > max_modid) ++ break; ++ ++ /* Synchronize with _dl_add_to_slotinfo. Ideally this would ++ be consume MO since we only need to order the accesses to ++ the next node after the read of the address and on most ++ hardware (other than alpha) a normal load would do that ++ because of the address dependency. */ ++ listp = atomic_load_acquire (&listp->next); + } +- while ((listp = listp->next) != NULL); ++ while (listp != NULL); + + /* This will be the new maximum generation counter. */ + dtv[0].counter = new_gen; +@@ -986,7 +1015,7 @@ _dl_add_to_slotinfo (struct link_map *l, bool do_add) + the first slot. */ + assert (idx == 0); + +- listp = prevp->next = (struct dtv_slotinfo_list *) ++ listp = (struct dtv_slotinfo_list *) + malloc (sizeof (struct dtv_slotinfo_list) + + TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); + if (listp == NULL) +@@ -1000,6 +1029,8 @@ cannot create TLS data structures")); + listp->next = NULL; + memset (listp->slotinfo, '\0', + TLS_SLOTINFO_SURPLUS * sizeof (struct dtv_slotinfo)); ++ /* Synchronize with _dl_update_slotinfo. */ ++ atomic_store_release (&prevp->next, listp); + } + + /* Add the information into the slotinfo data structure. */ +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0032-elf-Use-relaxed-atomics-for-racy-accesses-BZ-19329.patch b/meta/recipes-core/glibc/glibc/0032-elf-Use-relaxed-atomics-for-racy-accesses-BZ-19329.patch new file mode 100644 index 0000000000..eb8ef3161c --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0032-elf-Use-relaxed-atomics-for-racy-accesses-BZ-19329.patch @@ -0,0 +1,206 @@ +From f4f8f4d4e0f92488431b268c8cd9555730b9afe9 Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Wed, 30 Dec 2020 19:19:37 +0000 +Subject: [PATCH] elf: Use relaxed atomics for racy accesses [BZ #19329] + +This is a follow up patch to the fix for bug 19329. This adds relaxed +MO atomics to accesses that were previously data races but are now +race conditions, and where relaxed MO is sufficient. + +The race conditions all follow the pattern that the write is behind the +dlopen lock, but a read can happen concurrently (e.g. during tls access) +without holding the lock. For slotinfo entries the read value only +matters if it reads from a synchronized write in dlopen or dlclose, +otherwise the related dtv entry is not valid to access so it is fine +to leave it in an inconsistent state. The same applies for +GL(dl_tls_max_dtv_idx) and GL(dl_tls_generation), but there the +algorithm relies on the fact that the read of the last synchronized +write is an increasing value. + +Reviewed-by: Adhemerval Zanella +--- + elf/dl-close.c | 20 +++++++++++++------- + elf/dl-open.c | 5 ++++- + elf/dl-tls.c | 31 +++++++++++++++++++++++-------- + sysdeps/x86_64/dl-tls.c | 3 ++- + 4 files changed, 42 insertions(+), 17 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=f4f8f4d4e0f92488431b268c8cd9555730b9afe9] +Comment: Hunks from elf/dl-open.c and elf/dl-tls.c are refreshed due to offset change. +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/elf/dl-close.c b/elf/dl-close.c +index c51becd06b..3720e47dd1 100644 +--- a/elf/dl-close.c ++++ b/elf/dl-close.c +@@ -79,9 +79,10 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, + { + assert (old_map->l_tls_modid == idx); + +- /* Mark the entry as unused. */ +- listp->slotinfo[idx - disp].gen = GL(dl_tls_generation) + 1; +- listp->slotinfo[idx - disp].map = NULL; ++ /* Mark the entry as unused. These can be read concurrently. */ ++ atomic_store_relaxed (&listp->slotinfo[idx - disp].gen, ++ GL(dl_tls_generation) + 1); ++ atomic_store_relaxed (&listp->slotinfo[idx - disp].map, NULL); + } + + /* If this is not the last currently used entry no need to look +@@ -96,8 +97,8 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, + + if (listp->slotinfo[idx - disp].map != NULL) + { +- /* Found a new last used index. */ +- GL(dl_tls_max_dtv_idx) = idx; ++ /* Found a new last used index. This can be read concurrently. */ ++ atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), idx); + return true; + } + } +@@ -571,7 +572,9 @@ _dl_close_worker (struct link_map *map, bool force) + GL(dl_tls_dtv_slotinfo_list), 0, + imap->l_init_called)) + /* All dynamically loaded modules with TLS are unloaded. */ +- GL(dl_tls_max_dtv_idx) = GL(dl_tls_static_nelem); ++ /* Can be read concurrently. */ ++ atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), ++ GL(dl_tls_static_nelem)); + + if (imap->l_tls_offset != NO_TLS_OFFSET + && imap->l_tls_offset != FORCED_DYNAMIC_TLS_OFFSET) +@@ -769,8 +772,11 @@ _dl_close_worker (struct link_map *map, bool force) + /* If we removed any object which uses TLS bump the generation counter. */ + if (any_tls) + { +- if (__glibc_unlikely (++GL(dl_tls_generation) == 0)) ++ size_t newgen = GL(dl_tls_generation) + 1; ++ if (__glibc_unlikely (newgen == 0)) + _dl_fatal_printf ("TLS generation counter wrapped! Please report as described in "REPORT_BUGS_TO".\n"); ++ /* Can be read concurrently. */ ++ atomic_store_relaxed (&GL(dl_tls_generation), newgen); + + if (tls_free_end == GL(dl_tls_static_used)) + GL(dl_tls_static_used) = tls_free_start; +diff --git a/elf/dl-open.c b/elf/dl-open.c +index 09f0df7d38..bb79ef00f1 100644 +--- a/elf/dl-open.c ++++ b/elf/dl-open.c +@@ -387,9 +387,12 @@ + } + } + +- if (__builtin_expect (++GL(dl_tls_generation) == 0, 0)) ++ size_t newgen = GL(dl_tls_generation) + 1; ++ if (__glibc_unlikely (newgen == 0)) + _dl_fatal_printf (N_("\ + TLS generation counter wrapped! Please report this.")); ++ /* Can be read concurrently. */ ++ atomic_store_relaxed (&GL(dl_tls_generation), newgen); + + /* We need a second pass for static tls data, because + _dl_update_slotinfo must not be run while calls to +diff --git a/elf/dl-tls.c b/elf/dl-tls.c +index 94f3cdbae0..dc69cd984e 100644 +--- a/elf/dl-tls.c ++++ b/elf/dl-tls.c +@@ -96,7 +96,9 @@ + /* No gaps, allocate a new entry. */ + nogaps: + +- result = ++GL(dl_tls_max_dtv_idx); ++ result = GL(dl_tls_max_dtv_idx) + 1; ++ /* Can be read concurrently. */ ++ atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), result); + } + + return result; +@@ -279,10 +281,12 @@ + dtv_t *dtv; + size_t dtv_length; + ++ /* Relaxed MO, because the dtv size is later rechecked, not relied on. */ ++ size_t max_modid = atomic_load_relaxed (&GL(dl_tls_max_dtv_idx)); + /* We allocate a few more elements in the dtv than are needed for the + initial set of modules. This should avoid in most cases expansions + of the dtv. */ +- dtv_length = GL(dl_tls_max_dtv_idx) + DTV_SURPLUS; ++ dtv_length = max_modid + DTV_SURPLUS; + dtv = calloc (dtv_length + 2, sizeof (dtv_t)); + if (dtv != NULL) + { +@@ -687,7 +691,7 @@ + if (modid > max_modid) + break; + +- size_t gen = listp->slotinfo[cnt].gen; ++ size_t gen = atomic_load_relaxed (&listp->slotinfo[cnt].gen); + + if (gen > new_gen) + /* Not relevant. */ +@@ -699,7 +703,8 @@ + continue; + + /* If there is no map this means the entry is empty. */ +- struct link_map *map = listp->slotinfo[cnt].map; ++ struct link_map *map ++ = atomic_load_relaxed (&listp->slotinfo[cnt].map); + /* Check whether the current dtv array is large enough. */ + if (dtv[-1].counter < modid) + { +@@ -843,7 +848,12 @@ + { + dtv_t *dtv = THREAD_DTV (); + +- if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) ++ /* Update is needed if dtv[0].counter < the generation of the accessed ++ module. The global generation counter is used here as it is easier ++ to check. Synchronization for the relaxed MO access is guaranteed ++ by user code, see CONCURRENCY NOTES in _dl_update_slotinfo. */ ++ size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); ++ if (__glibc_unlikely (dtv[0].counter != gen)) + return update_get_addr (GET_ADDR_PARAM); + + void *p = dtv[GET_ADDR_MODULE].pointer.val; +@@ -866,7 +876,10 @@ + return NULL; + + dtv_t *dtv = THREAD_DTV (); +- if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) ++ /* This may be called without holding the GL(dl_load_lock). Reading ++ arbitrary gen value is fine since this is best effort code. */ ++ size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); ++ if (__glibc_unlikely (dtv[0].counter != gen)) + { + /* This thread's DTV is not completely current, + but it might already cover this module. */ +@@ -961,7 +974,9 @@ + /* Add the information into the slotinfo data structure. */ + if (do_add) + { +- listp->slotinfo[idx].map = l; +- listp->slotinfo[idx].gen = GL(dl_tls_generation) + 1; ++ /* Can be read concurrently. See _dl_update_slotinfo. */ ++ atomic_store_relaxed (&listp->slotinfo[idx].map, l); ++ atomic_store_relaxed (&listp->slotinfo[idx].gen, ++ GL(dl_tls_generation) + 1); + } + } + +diff --git a/sysdeps/x86_64/dl-tls.c b/sysdeps/x86_64/dl-tls.c +index 6595f6615b..24ef560b71 100644 +--- a/sysdeps/x86_64/dl-tls.c ++++ b/sysdeps/x86_64/dl-tls.c +@@ -40,7 +40,8 @@ __tls_get_addr_slow (GET_ADDR_ARGS) + { + dtv_t *dtv = THREAD_DTV (); + +- if (__glibc_unlikely (dtv[0].counter != GL(dl_tls_generation))) ++ size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); ++ if (__glibc_unlikely (dtv[0].counter != gen)) + return update_get_addr (GET_ADDR_PARAM); + + return tls_get_addr_tail (GET_ADDR_PARAM, dtv, NULL); +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0033-elf-Add-test-case-for-BZ-19329.patch b/meta/recipes-core/glibc/glibc/0033-elf-Add-test-case-for-BZ-19329.patch new file mode 100644 index 0000000000..f22e52ea99 --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0033-elf-Add-test-case-for-BZ-19329.patch @@ -0,0 +1,144 @@ +From 9d0e30329c23b5ad736fda3f174208c25970dbce Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Tue, 13 Dec 2016 12:28:41 +0000 +Subject: [PATCH] elf: Add test case for [BZ #19329] + +Test concurrent dlopen and pthread_create when the loaded modules have +TLS. This triggers dl-tls assertion failures more reliably than the +nptl/tst-stack4 test. + +The dlopened module has 100 DT_NEEDED dependencies with TLS, they were +reused from an existing TLS test. The number of created threads during +dlopen depends on filesystem speed and hardware, but at most 3 threads +are alive at a time to limit resource usage. + +Reviewed-by: Adhemerval Zanella +--- + elf/Makefile | 9 ++++-- + elf/tst-tls21.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++ + elf/tst-tls21mod.c | 1 + + 3 files changed, 76 insertions(+), 2 deletions(-) + create mode 100644 elf/tst-tls21.c + create mode 100644 elf/tst-tls21mod.c +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=9d0e30329c23b5ad736fda3f174208c25970dbce] +Comment: Hunks from elf/Makefile are refreshed as per glibc 2.31 codebase. +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/elf/Makefile b/elf/Makefile +index d3e909637a..3241cb6046 100644 +--- a/elf/Makefile ++++ b/elf/Makefile +@@ -201,7 +201,7 @@ + tst-unwind-ctor tst-unwind-main tst-audit13 \ + tst-sonamemove-link tst-sonamemove-dlopen tst-dlopen-tlsmodid \ + tst-dlopen-self tst-auditmany tst-initfinilazyfail tst-dlopenfail \ +- tst-dlopenfail-2 ++ tst-dlopenfail-2 tst-tls21 + # reldep9 + tests-internal += loadtest unload unload2 circleload1 \ + neededtest neededtest2 neededtest3 neededtest4 \ +@@ -312,7 +312,7 @@ + tst-auditmanymod7 tst-auditmanymod8 tst-auditmanymod9 \ + tst-initlazyfailmod tst-finilazyfailmod \ + tst-dlopenfailmod1 tst-dlopenfaillinkmod tst-dlopenfailmod2 \ +- tst-dlopenfailmod3 tst-ldconfig-ld-mod ++ tst-dlopenfailmod3 tst-ldconfig-ld-mod tst-tls21mod + # Most modules build with _ISOMAC defined, but those filtered out + # depend on internal headers. + modules-names-tests = $(filter-out ifuncmod% tst-libc_dlvsym-dso tst-tlsmod%,\ +@@ -1697,5 +1697,10 @@ + $(objpfx)tst-dlopen-nodelete-reloc-mod16.so + LDFLAGS-tst-dlopen-nodelete-reloc-mod17.so = -Wl,--no-as-needed + ++# Reuses tst-tls-many-dynamic-modules ++$(objpfx)tst-tls21: $(libdl) $(shared-thread-library) ++$(objpfx)tst-tls21.out: $(objpfx)tst-tls21mod.so ++$(objpfx)tst-tls21mod.so: $(tst-tls-many-dynamic-modules:%=$(objpfx)%.so) ++ + $(objpfx)tst-ldconfig-ld_so_conf-update.out: $(objpfx)tst-ldconfig-ld-mod.so + $(objpfx)tst-ldconfig-ld_so_conf-update: $(libdl) +diff --git a/elf/tst-tls21.c b/elf/tst-tls21.c +new file mode 100644 +index 0000000000..560bf5813a +--- /dev/null ++++ b/elf/tst-tls21.c +@@ -0,0 +1,68 @@ ++/* Test concurrent dlopen and pthread_create: BZ 19329. ++ Copyright (C) 2021 Free Software Foundation, Inc. ++ This file is part of the GNU C Library. ++ ++ The GNU C Library is free software; you can redistribute it and/or ++ modify it under the terms of the GNU Lesser General Public ++ License as published by the Free Software Foundation; either ++ version 2.1 of the License, or (at your option) any later version. ++ ++ The GNU C Library is distributed in the hope that it will be useful, ++ but WITHOUT ANY WARRANTY; without even the implied warranty of ++ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ++ Lesser General Public License for more details. ++ ++ You should have received a copy of the GNU Lesser General Public ++ License along with the GNU C Library; if not, see ++ . */ ++ ++#include ++#include ++#include ++#include ++#include ++#include ++ ++#define THREADS 10000 ++ ++static atomic_int done; ++ ++static void * ++start (void *a) ++{ ++ /* Load a module with many dependencies that each have TLS. */ ++ xdlopen ("tst-tls21mod.so", RTLD_LAZY); ++ atomic_store_explicit (&done, 1, memory_order_release); ++ return 0; ++} ++ ++static void * ++nop (void *a) ++{ ++ return 0; ++} ++ ++static int ++do_test (void) ++{ ++ pthread_t t1, t2; ++ int i; ++ ++ /* Load a module with lots of dependencies and TLS. */ ++ t1 = xpthread_create (0, start, 0); ++ ++ /* Concurrently create lots of threads until dlopen is observably done. */ ++ for (i = 0; i < THREADS; i++) ++ { ++ if (atomic_load_explicit (&done, memory_order_acquire) != 0) ++ break; ++ t2 = xpthread_create (0, nop, 0); ++ xpthread_join (t2); ++ } ++ ++ xpthread_join (t1); ++ printf ("threads created during dlopen: %d\n", i); ++ return 0; ++} ++ ++#include +diff --git a/elf/tst-tls21mod.c b/elf/tst-tls21mod.c +new file mode 100644 +index 0000000000..206ece4fb3 +--- /dev/null ++++ b/elf/tst-tls21mod.c +@@ -0,0 +1 @@ ++int __thread x; +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0034-elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch b/meta/recipes-core/glibc/glibc/0034-elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch new file mode 100644 index 0000000000..a87afe3230 --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0034-elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch @@ -0,0 +1,180 @@ +From ba33937be210da5d07f7f01709323743f66011ce Mon Sep 17 00:00:00 2001 +From: Adhemerval Zanella +Date: Fri, 25 Jun 2021 10:54:12 -0300 +Subject: [PATCH] elf: Fix DTV gap reuse logic (BZ #27135) + +This is updated version of the 572bd547d57a (reverted by 40ebfd016ad2) +that fixes the _dl_next_tls_modid issues. + +This issue with 572bd547d57a patch is the DTV entry will be only +update on dl_open_worker() with the update_tls_slotinfo() call after +all dependencies are being processed by _dl_map_object_deps(). However +_dl_map_object_deps() itself might call _dl_next_tls_modid(), and since +the _dl_tls_dtv_slotinfo_list::map is not yet set the entry will be +wrongly reused. + +This patch fixes by renaming the _dl_next_tls_modid() function to +_dl_assign_tls_modid() and by passing the link_map so it can set +the slotinfo value so a subsequente _dl_next_tls_modid() call will +see the entry as allocated. + +The intermediary value is cleared up on remove_slotinfo() for the case +a library fails to load with RTLD_NOW. + +This patch fixes BZ #27135. + +Checked on x86_64-linux-gnu. + +Reviewed-by: Szabolcs Nagy +--- + elf/dl-close.c | 8 +- + elf/dl-load.c | 2 +- + elf/dl-open.c | 10 -- + elf/dl-tls.c | 17 +-- + elf/rtld.c | 2 +- + sysdeps/generic/ldsodefs.h | 4 +- + 6 files changed, 349 insertions(+), 33 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=ba33937be210da5d07f7f01709323743f66011ce] +Comment: Removed hunks those were related to test. Hunk from elf/rtld.c is refreshed. +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/elf/dl-close.c b/elf/dl-close.c +index 3720e47dd1..f39001cab9 100644 +--- a/elf/dl-close.c ++++ b/elf/dl-close.c +@@ -77,8 +77,6 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, + object that wasn't fully set up. */ + if (__glibc_likely (old_map != NULL)) + { +- assert (old_map->l_tls_modid == idx); +- + /* Mark the entry as unused. These can be read concurrently. */ + atomic_store_relaxed (&listp->slotinfo[idx - disp].gen, + GL(dl_tls_generation) + 1); +@@ -88,7 +86,11 @@ remove_slotinfo (size_t idx, struct dtv_slotinfo_list *listp, size_t disp, + /* If this is not the last currently used entry no need to look + further. */ + if (idx != GL(dl_tls_max_dtv_idx)) +- return true; ++ { ++ /* There is an unused dtv entry in the middle. */ ++ GL(dl_tls_dtv_gaps) = true; ++ return true; ++ } + } + + while (idx - disp > (disp == 0 ? 1 + GL(dl_tls_static_nelem) : 0)) +diff --git a/elf/dl-load.c b/elf/dl-load.c +index a08df001af..650e4edc35 100644 +--- a/elf/dl-load.c ++++ b/elf/dl-load.c +@@ -1498,7 +1498,7 @@ cannot enable executable stack as shared object requires"); + not set up TLS data structures, so don't use them now. */ + || __glibc_likely (GL(dl_tls_dtv_slotinfo_list) != NULL))) + /* Assign the next available module ID. */ +- l->l_tls_modid = _dl_next_tls_modid (); ++ _dl_assign_tls_modid (l); + + #ifdef DL_AFTER_LOAD + DL_AFTER_LOAD (l); +diff --git a/elf/dl-open.c b/elf/dl-open.c +index a066f39bd0..d2240d8747 100644 +--- a/elf/dl-open.c ++++ b/elf/dl-open.c +@@ -899,16 +899,6 @@ no more namespaces available for dlmopen()")); + state if relocation failed, for example. */ + if (args.map) + { +- /* Maybe some of the modules which were loaded use TLS. +- Since it will be removed in the following _dl_close call +- we have to mark the dtv array as having gaps to fill the +- holes. This is a pessimistic assumption which won't hurt +- if not true. There is no need to do this when we are +- loading the auditing DSOs since TLS has not yet been set +- up. */ +- if ((mode & __RTLD_AUDIT) == 0) +- GL(dl_tls_dtv_gaps) = true; +- + _dl_close_worker (args.map, true); + + /* All l_nodelete_pending objects should have been deleted +diff --git a/elf/dl-tls.c b/elf/dl-tls.c +index 2b5161d10a..423e380f7c 100644 +--- a/elf/dl-tls.c ++++ b/elf/dl-tls.c +@@ -126,8 +126,8 @@ oom (void) + } + + +-size_t +-_dl_next_tls_modid (void) ++void ++_dl_assign_tls_modid (struct link_map *l) + { + size_t result; + +@@ -157,7 +157,11 @@ _dl_next_tls_modid (void) + } + + if (result - disp < runp->len) +- break; ++ { ++ /* Mark the entry as used, so any dependency see it. */ ++ atomic_store_relaxed (&runp->slotinfo[result - disp].map, l); ++ break; ++ } + + disp += runp->len; + } +@@ -184,17 +188,14 @@ _dl_next_tls_modid (void) + atomic_store_relaxed (&GL(dl_tls_max_dtv_idx), result); + } + +- return result; ++ l->l_tls_modid = result; + } + + + size_t + _dl_count_modids (void) + { +- /* It is rare that we have gaps; see elf/dl-open.c (_dl_open) where +- we fail to load a module and unload it leaving a gap. If we don't +- have gaps then the number of modids is the current maximum so +- return that. */ ++ /* The count is the max unless dlclose or failed dlopen created gaps. */ + if (__glibc_likely (!GL(dl_tls_dtv_gaps))) + return GL(dl_tls_max_dtv_idx); + +diff --git a/elf/rtld.c b/elf/rtld.c +index e3fb2a5b2a..d733359eaf 100644 +--- a/elf/rtld.c ++++ b/elf/rtld.c +@@ -1612,7 +1612,7 @@ + /* Add the dynamic linker to the TLS list if it also uses TLS. */ + if (GL(dl_rtld_map).l_tls_blocksize != 0) + /* Assign a module ID. Do this before loading any audit modules. */ +- GL(dl_rtld_map).l_tls_modid = _dl_next_tls_modid (); ++ _dl_assign_tls_modid (&GL(dl_rtld_map)); + + /* If we have auditing DSOs to load, do it now. */ + bool need_security_init = true; +diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h +index 176394de4d..9c15259236 100644 +--- a/sysdeps/generic/ldsodefs.h ++++ b/sysdeps/generic/ldsodefs.h +@@ -1171,8 +1171,8 @@ extern ElfW(Addr) _dl_sysdep_start (void **start_argptr, + extern void _dl_sysdep_start_cleanup (void) attribute_hidden; + + +-/* Determine next available module ID. */ +-extern size_t _dl_next_tls_modid (void) attribute_hidden; ++/* Determine next available module ID and set the L l_tls_modid. */ ++extern void _dl_assign_tls_modid (struct link_map *l) attribute_hidden; + + /* Count the modules with TLS segments. */ + extern size_t _dl_count_modids (void) attribute_hidden; +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0035-x86_64-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch b/meta/recipes-core/glibc/glibc/0035-x86_64-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch new file mode 100644 index 0000000000..899111b118 --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0035-x86_64-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch @@ -0,0 +1,56 @@ +From 8f7e09f4dbdb5c815a18b8285fbc5d5d7bc17d86 Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Thu, 11 Feb 2021 11:29:23 +0000 +Subject: [PATCH] x86_64: Avoid lazy relocation of tlsdesc [BZ #27137] + +Lazy tlsdesc relocation is racy because the static tls optimization and +tlsdesc management operations are done without holding the dlopen lock. + +This similar to the commit b7cf203b5c17dd6d9878537d41e0c7cc3d270a67 +for aarch64, but it fixes a different race: bug 27137. + +Another issue is that ld auditing ignores DT_BIND_NOW and thus tries to +relocate tlsdesc lazily, but that does not work in a BIND_NOW module +due to missing DT_TLSDESC_PLT. Unconditionally relocating tlsdesc at +load time fixes this bug 27721 too. +--- + sysdeps/x86_64/dl-machine.h | 19 ++++++++++++++----- + 1 file changed, 14 insertions(+), 5 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=8f7e09f4dbdb5c815a18b8285fbc5d5d7bc17d86] +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/sysdeps/x86_64/dl-machine.h b/sysdeps/x86_64/dl-machine.h +index 103eee6c3f..9a876a371e 100644 +--- a/sysdeps/x86_64/dl-machine.h ++++ b/sysdeps/x86_64/dl-machine.h +@@ -570,12 +570,21 @@ elf_machine_lazy_rel (struct link_map *map, + } + else if (__glibc_likely (r_type == R_X86_64_TLSDESC)) + { +- struct tlsdesc volatile * __attribute__((__unused__)) td = +- (struct tlsdesc volatile *)reloc_addr; ++ const Elf_Symndx symndx = ELFW (R_SYM) (reloc->r_info); ++ const ElfW (Sym) *symtab = (const void *)D_PTR (map, l_info[DT_SYMTAB]); ++ const ElfW (Sym) *sym = &symtab[symndx]; ++ const struct r_found_version *version = NULL; + +- td->arg = (void*)reloc; +- td->entry = (void*)(D_PTR (map, l_info[ADDRIDX (DT_TLSDESC_PLT)]) +- + map->l_addr); ++ if (map->l_info[VERSYMIDX (DT_VERSYM)] != NULL) ++ { ++ const ElfW (Half) *vernum = ++ (const void *)D_PTR (map, l_info[VERSYMIDX (DT_VERSYM)]); ++ version = &map->l_versions[vernum[symndx] & 0x7fff]; ++ } ++ ++ /* Always initialize TLS descriptors completely at load time, in ++ case static TLS is allocated for it that requires locking. */ ++ elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc); + } + else if (__glibc_unlikely (r_type == R_X86_64_IRELATIVE)) + { +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch b/meta/recipes-core/glibc/glibc/0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch new file mode 100644 index 0000000000..ad0a1147aa --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch @@ -0,0 +1,124 @@ +From ddcacd91cc10ff92d6201eda87047d029c14158d Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Thu, 11 Feb 2021 11:40:11 +0000 +Subject: [PATCH] i386: Avoid lazy relocation of tlsdesc [BZ #27137] + +Lazy tlsdesc relocation is racy because the static tls optimization and +tlsdesc management operations are done without holding the dlopen lock. + +This similar to the commit b7cf203b5c17dd6d9878537d41e0c7cc3d270a67 +for aarch64, but it fixes a different race: bug 27137. + +On i386 the code is a bit more complicated than on x86_64 because both +rel and rela relocs are supported. +--- + sysdeps/i386/dl-machine.h | 76 ++++++++++++++++++--------------------- + 1 file changed, 34 insertions(+), 42 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=ddcacd91cc10ff92d6201eda87047d029c14158d] +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/sysdeps/i386/dl-machine.h b/sysdeps/i386/dl-machine.h +index 23e9cc3bfb..590b41d8d7 100644 +--- a/sysdeps/i386/dl-machine.h ++++ b/sysdeps/i386/dl-machine.h +@@ -688,50 +688,32 @@ elf_machine_lazy_rel (struct link_map *map, + } + else if (__glibc_likely (r_type == R_386_TLS_DESC)) + { +- struct tlsdesc volatile * __attribute__((__unused__)) td = +- (struct tlsdesc volatile *)reloc_addr; +- +- /* Handle relocations that reference the local *ABS* in a simple +- way, so as to preserve a potential addend. */ +- if (ELF32_R_SYM (reloc->r_info) == 0) +- td->entry = _dl_tlsdesc_resolve_abs_plus_addend; +- /* Given a known-zero addend, we can store a pointer to the +- reloc in the arg position. */ +- else if (td->arg == 0) +- { +- td->arg = (void*)reloc; +- td->entry = _dl_tlsdesc_resolve_rel; +- } +- else +- { +- /* We could handle non-*ABS* relocations with non-zero addends +- by allocating dynamically an arg to hold a pointer to the +- reloc, but that sounds pointless. */ +- const Elf32_Rel *const r = reloc; +- /* The code below was borrowed from elf_dynamic_do_rel(). */ +- const ElfW(Sym) *const symtab = +- (const void *) D_PTR (map, l_info[DT_SYMTAB]); ++ const Elf32_Rel *const r = reloc; ++ /* The code below was borrowed from elf_dynamic_do_rel(). */ ++ const ElfW(Sym) *const symtab = ++ (const void *) D_PTR (map, l_info[DT_SYMTAB]); + ++ /* Always initialize TLS descriptors completely at load time, in ++ case static TLS is allocated for it that requires locking. */ + # ifdef RTLD_BOOTSTRAP +- /* The dynamic linker always uses versioning. */ +- assert (map->l_info[VERSYMIDX (DT_VERSYM)] != NULL); ++ /* The dynamic linker always uses versioning. */ ++ assert (map->l_info[VERSYMIDX (DT_VERSYM)] != NULL); + # else +- if (map->l_info[VERSYMIDX (DT_VERSYM)]) ++ if (map->l_info[VERSYMIDX (DT_VERSYM)]) + # endif +- { +- const ElfW(Half) *const version = +- (const void *) D_PTR (map, l_info[VERSYMIDX (DT_VERSYM)]); +- ElfW(Half) ndx = version[ELFW(R_SYM) (r->r_info)] & 0x7fff; +- elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], +- &map->l_versions[ndx], +- (void *) (l_addr + r->r_offset), skip_ifunc); +- } ++ { ++ const ElfW(Half) *const version = ++ (const void *) D_PTR (map, l_info[VERSYMIDX (DT_VERSYM)]); ++ ElfW(Half) ndx = version[ELFW(R_SYM) (r->r_info)] & 0x7fff; ++ elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], ++ &map->l_versions[ndx], ++ (void *) (l_addr + r->r_offset), skip_ifunc); ++ } + # ifndef RTLD_BOOTSTRAP +- else +- elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], NULL, +- (void *) (l_addr + r->r_offset), skip_ifunc); ++ else ++ elf_machine_rel (map, r, &symtab[ELFW(R_SYM) (r->r_info)], NULL, ++ (void *) (l_addr + r->r_offset), skip_ifunc); + # endif +- } + } + else if (__glibc_unlikely (r_type == R_386_IRELATIVE)) + { +@@ -758,11 +740,21 @@ elf_machine_lazy_rela (struct link_map *map, + ; + else if (__glibc_likely (r_type == R_386_TLS_DESC)) + { +- struct tlsdesc volatile * __attribute__((__unused__)) td = +- (struct tlsdesc volatile *)reloc_addr; ++ const Elf_Symndx symndx = ELFW (R_SYM) (reloc->r_info); ++ const ElfW (Sym) *symtab = (const void *)D_PTR (map, l_info[DT_SYMTAB]); ++ const ElfW (Sym) *sym = &symtab[symndx]; ++ const struct r_found_version *version = NULL; ++ ++ if (map->l_info[VERSYMIDX (DT_VERSYM)] != NULL) ++ { ++ const ElfW (Half) *vernum = ++ (const void *)D_PTR (map, l_info[VERSYMIDX (DT_VERSYM)]); ++ version = &map->l_versions[vernum[symndx] & 0x7fff]; ++ } + +- td->arg = (void*)reloc; +- td->entry = _dl_tlsdesc_resolve_rela; ++ /* Always initialize TLS descriptors completely at load time, in ++ case static TLS is allocated for it that requires locking. */ ++ elf_machine_rela (map, reloc, sym, version, reloc_addr, skip_ifunc); + } + else if (__glibc_unlikely (r_type == R_386_IRELATIVE)) + { +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc/0037-Avoid-deadlock-between-pthread_create-and-ctors.patch b/meta/recipes-core/glibc/glibc/0037-Avoid-deadlock-between-pthread_create-and-ctors.patch new file mode 100644 index 0000000000..7a10131bad --- /dev/null +++ b/meta/recipes-core/glibc/glibc/0037-Avoid-deadlock-between-pthread_create-and-ctors.patch @@ -0,0 +1,276 @@ +From 83b5323261bb72313bffcf37476c1b8f0847c736 Mon Sep 17 00:00:00 2001 +From: Szabolcs Nagy +Date: Wed, 15 Sep 2021 15:16:19 +0100 +Subject: [PATCH] elf: Avoid deadlock between pthread_create and ctors [BZ + #28357] + +The fix for bug 19329 caused a regression such that pthread_create can +deadlock when concurrent ctors from dlopen are waiting for it to finish. +Use a new GL(dl_load_tls_lock) in pthread_create that is not taken +around ctors in dlopen. + +The new lock is also used in __tls_get_addr instead of GL(dl_load_lock). + +The new lock is held in _dl_open_worker and _dl_close_worker around +most of the logic before/after the init/fini routines. When init/fini +routines are running then TLS is in a consistent, usable state. +In _dl_open_worker the new lock requires catching and reraising dlopen +failures that happen in the critical section. + +The new lock is reinitialized in a fork child, to keep the existing +behaviour and it is kept recursive in case malloc interposition or TLS +access from signal handlers can retake it. It is not obvious if this +is necessary or helps, but avoids changing the preexisting behaviour. + +The new lock may be more appropriate for dl_iterate_phdr too than +GL(dl_load_write_lock), since TLS state of an incompletely loaded +module may be accessed. If the new lock can replace the old one, +that can be a separate change. + +Fixes bug 28357. + +Reviewed-by: Adhemerval Zanella +--- + elf/dl-close.c | 6 ++ + elf/dl-open.c | 35 ++++++++- + elf/dl-support.c | 7 ++ + elf/dl-tls.c | 16 ++--- + elf/rtld.c | 1 + + sysdeps/nptl/fork.c | 3 + + sysdeps/generic/ldsodefs.h | 9 ++- + 10 files changed, 235 insertions(+), 12 deletions(-) +--- +Upstream-Status: Backport [https://sourceware.org/git/?p=glibc.git;a=patch;h=024a7640ab9ecea80e527f4e4d7f7a1868e952c5] +Comment: This patch is refreshed for glibc 2.31. In upstream glibc 2.34 multiple src files are shuffled, updated this patch as per the code present in glibc 2.31. Removed test case. +Signed-off-by: Akash Hadke +Signed-off-by: Akash Hadke +--- +diff --git a/elf/dl-close.c b/elf/dl-close.c +index 93ff5c96e9..cfe0f1c0c9 100644 +--- a/elf/dl-close.c ++++ b/elf/dl-close.c +@@ -551,6 +551,9 @@ + size_t tls_free_end; + tls_free_start = tls_free_end = NO_TLS_OFFSET; + ++ /* Protects global and module specitic TLS state. */ ++ __rtld_lock_lock_recursive (GL(dl_load_tls_lock)); ++ + /* We modify the list of loaded objects. */ + __rtld_lock_lock_recursive (GL(dl_load_write_lock)); + +@@ -786,6 +789,9 @@ + GL(dl_tls_static_used) = tls_free_start; + } + ++ /* TLS is cleaned up for the unloaded modules. */ ++ __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); ++ + #ifdef SHARED + /* Auditing checkpoint: we have deleted all objects. */ + if (__glibc_unlikely (do_audit)) +diff --git a/elf/dl-open.c b/elf/dl-open.c +index 5295e931b0..6ea5dd2457 100644 +--- a/elf/dl-open.c ++++ b/elf/dl-open.c +@@ -57,6 +57,9 @@ + (non-negative). */ + unsigned int original_global_scope_pending_adds; + ++ /* Set to true if the end of dl_open_worker_begin was reached. */ ++ bool worker_continue; ++ + /* Original parameters to the program and the current environment. */ + int argc; + char **argv; +@@ -473,7 +473,7 @@ + } + + static void +-dl_open_worker (void *a) ++dl_open_worker_begin (void *a) + { + struct dl_open_args *args = a; + const char *file = args->file; +@@ -747,6 +747,36 @@ + if (mode & RTLD_GLOBAL) + add_to_global_resize (new); + ++ args->worker_continue = true; ++} ++ ++static void ++dl_open_worker (void *a) ++{ ++ struct dl_open_args *args = a; ++ ++ args->worker_continue = false; ++ ++ { ++ /* Protects global and module specific TLS state. */ ++ __rtld_lock_lock_recursive (GL(dl_load_tls_lock)); ++ ++ struct dl_exception ex; ++ int err = _dl_catch_exception (&ex, dl_open_worker_begin, args); ++ ++ __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); ++ ++ if (__glibc_unlikely (ex.errstring != NULL)) ++ /* Reraise the error. */ ++ _dl_signal_exception (err, &ex, NULL); ++ } ++ ++ if (!args->worker_continue) ++ return; ++ ++ int mode = args->mode; ++ struct link_map *new = args->map; ++ + /* Run the initializer functions of new objects. Temporarily + disable the exception handler, so that lazy binding failures are + fatal. */ +diff --git a/elf/dl-support.c b/elf/dl-support.c +index 02e2ed72f5..d99c1f1d62 100644 +--- a/elf/dl-support.c ++++ b/elf/dl-support.c +@@ -219,6 +219,13 @@ + list of loaded objects while an object is added to or removed from + that list. */ + __rtld_lock_define_initialized_recursive (, _dl_load_write_lock) ++/* This lock protects global and module specific TLS related data. ++ E.g. it is held in dlopen and dlclose when GL(dl_tls_generation), ++ GL(dl_tls_max_dtv_idx) or GL(dl_tls_dtv_slotinfo_list) are ++ accessed and when TLS related relocations are processed for a ++ module. It was introduced to keep pthread_create accessing TLS ++ state that is being set up. */ ++__rtld_lock_define_initialized_recursive (, _dl_load_tls_lock) + + + #ifdef HAVE_AUX_VECTOR +diff --git a/elf/dl-tls.c b/elf/dl-tls.c +index d554ae4497..9260d2d696 100644 +--- a/elf/dl-tls.c ++++ b/elf/dl-tls.c +@@ -443,7 +443,7 @@ + size_t maxgen = 0; + + /* Protects global dynamic TLS related state. */ +- __rtld_lock_lock_recursive (GL(dl_load_lock)); ++ __rtld_lock_lock_recursive (GL(dl_load_tls_lock)); + + /* Check if the current dtv is big enough. */ + if (dtv[-1].counter < GL(dl_tls_max_dtv_idx)) +@@ -517,7 +517,7 @@ + listp = listp->next; + assert (listp != NULL); + } +- __rtld_lock_unlock_recursive (GL(dl_load_lock)); ++ __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); + + /* The DTV version is up-to-date now. */ + dtv[0].counter = maxgen; +@@ -656,7 +656,7 @@ + + Here the dtv needs to be updated to new_gen generation count. + +- This code may be called during TLS access when GL(dl_load_lock) ++ This code may be called during TLS access when GL(dl_load_tls_lock) + is not held. In that case the user code has to synchronize with + dlopen and dlclose calls of relevant modules. A module m is + relevant if the generation of m <= new_gen and dlclose of m is +@@ -778,11 +778,11 @@ + if (__glibc_unlikely (the_map->l_tls_offset + != FORCED_DYNAMIC_TLS_OFFSET)) + { +- __rtld_lock_lock_recursive (GL(dl_load_lock)); ++ __rtld_lock_lock_recursive (GL(dl_load_tls_lock)); + if (__glibc_likely (the_map->l_tls_offset == NO_TLS_OFFSET)) + { + the_map->l_tls_offset = FORCED_DYNAMIC_TLS_OFFSET; +- __rtld_lock_unlock_recursive (GL(dl_load_lock)); ++ __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); + } + else if (__glibc_likely (the_map->l_tls_offset + != FORCED_DYNAMIC_TLS_OFFSET)) +@@ -794,7 +794,7 @@ + #else + # error "Either TLS_TCB_AT_TP or TLS_DTV_AT_TP must be defined" + #endif +- __rtld_lock_unlock_recursive (GL(dl_load_lock)); ++ __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); + + dtv[GET_ADDR_MODULE].pointer.to_free = NULL; + dtv[GET_ADDR_MODULE].pointer.val = p; +@@ -802,7 +802,7 @@ + return (char *) p + GET_ADDR_OFFSET; + } + else +- __rtld_lock_unlock_recursive (GL(dl_load_lock)); ++ __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); + } + struct dtv_pointer result = allocate_and_init (the_map); + dtv[GET_ADDR_MODULE].pointer = result; +@@ -873,7 +873,7 @@ + return NULL; + + dtv_t *dtv = THREAD_DTV (); +- /* This may be called without holding the GL(dl_load_lock). Reading ++ /* This may be called without holding the GL(dl_load_tls_lock). Reading + arbitrary gen value is fine since this is best effort code. */ + size_t gen = atomic_load_relaxed (&GL(dl_tls_generation)); + if (__glibc_unlikely (dtv[0].counter != gen)) +diff --git a/elf/rtld.c b/elf/rtld.c +index 8d2bba3d43..9642eb9c92 100644 +--- a/elf/rtld.c ++++ b/elf/rtld.c +@@ -283,6 +283,7 @@ + #ifdef _LIBC_REENTRANT + ._dl_load_lock = _RTLD_LOCK_RECURSIVE_INITIALIZER, + ._dl_load_write_lock = _RTLD_LOCK_RECURSIVE_INITIALIZER, ++ ._dl_load_tls_lock = _RTLD_LOCK_RECURSIVE_INITIALIZER, + #endif + ._dl_nns = 1, + ._dl_ns = +diff --git a/sysdeps/nptl/fork.c b/sysdeps/nptl/fork.c +index c471f7b15f..021691b9b7 100644 +--- a/sysdeps/nptl/fork.c ++++ b/sysdeps/nptl/fork.c +@@ -125,6 +125,9 @@ + /* Reset the lock the dynamic loader uses to protect its data. */ + __rtld_lock_initialize (GL(dl_load_lock)); + ++ /* Reset the lock protecting dynamic TLS related data. */ ++ __rtld_lock_initialize (GL(dl_load_tls_lock)); ++ + /* Run the handlers registered for the child. */ + __run_fork_handlers (atfork_run_child, multiple_threads); + } +diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h +index d49529da0d..9ec1511bb0 100644 +--- a/sysdeps/generic/ldsodefs.h ++++ b/sysdeps/generic/ldsodefs.h +@@ -369,6 +369,13 @@ + list of loaded objects while an object is added to or removed + from that list. */ + __rtld_lock_define_recursive (EXTERN, _dl_load_write_lock) ++ /* This lock protects global and module specific TLS related data. ++ E.g. it is held in dlopen and dlclose when GL(dl_tls_generation), ++ GL(dl_tls_max_dtv_idx) or GL(dl_tls_dtv_slotinfo_list) are ++ accessed and when TLS related relocations are processed for a ++ module. It was introduced to keep pthread_create accessing TLS ++ state that is being set up. */ ++ __rtld_lock_define_recursive (EXTERN, _dl_load_tls_lock) + + /* Incremented whenever something may have been added to dl_loaded. */ + EXTERN unsigned long long _dl_load_adds; +@@ -1153,7 +1160,7 @@ + + /* Add module to slot information data. If DO_ADD is false, only the + required memory is allocated. Must be called with GL +- (dl_load_lock) acquired. If the function has already been called ++ (dl_load_tls_lock) acquired. If the function has already been called + for the link map L with !do_add, then this function will not raise + an exception, otherwise it is possible that it encounters a memory + allocation failure. */ +-- +2.27.0 diff --git a/meta/recipes-core/glibc/glibc_2.31.bb b/meta/recipes-core/glibc/glibc_2.31.bb index 3a3586f1b9..4a545cb97d 100644 --- a/meta/recipes-core/glibc/glibc_2.31.bb +++ b/meta/recipes-core/glibc/glibc_2.31.bb @@ -70,6 +70,14 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \ file://CVE-2021-33574_1.patch \ file://CVE-2021-33574_2.patch \ file://CVE-2021-38604.patch \ + file://0030-elf-Refactor_dl_update-slotinfo-to-avoid-use-after-free.patch \ + file://0031-elf-Fix-data-races-in-pthread_create-and-TLS-access-BZ-19329.patch \ + file://0032-elf-Use-relaxed-atomics-for-racy-accesses-BZ-19329.patch \ + file://0033-elf-Add-test-case-for-BZ-19329.patch \ + file://0034-elf-Fix-DTV-gap-reuse-logic-BZ-27135.patch \ + file://0035-x86_64-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch \ + file://0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch \ + file://0037-Avoid-deadlock-between-pthread_create-and-ctors.patch \ " S = "${WORKDIR}/git" B = "${WORKDIR}/build-${TARGET_SYS}"