From patchwork Fri Apr 7 03:07:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ChenQi X-Patchwork-Id: 22361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6182BC77B6C for ; Fri, 7 Apr 2023 03:07:38 +0000 (UTC) Received: from mx0a-0064b401.pphosted.com (mx0a-0064b401.pphosted.com [205.220.166.238]) by mx.groups.io with SMTP id smtpd.web11.70.1680836849232696229 for ; Thu, 06 Apr 2023 20:07:29 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="body hash did not verify" header.i=@windriver.com header.s=pps06212021 header.b=J72a1f4s; spf=permerror, err=parse error for token &{10 18 %{ir}.%{v}.%{d}.spf.has.pphosted.com}: invalid domain name (domain: windriver.com, ip: 205.220.166.238, mailfrom: prvs=3461ef5591=qi.chen@windriver.com) Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3372iTZj004635 for ; Thu, 6 Apr 2023 20:07:29 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=PPS06212021; bh=sJlbIir+dgUCJS3A7QC3N1oMGwsBMT48YvNL3LIquwU=; b=J72a1f4sjhyBLb2wGbzczdNUJYC+SGfyQS4YoAvzEfJn/1zmLnmLTBO5wrYXzJsqxIkG JN6aMbkH5kgzXkrl2vJoKOoJ9M6mxlfvHRPd8+wPxhx+X1Du11fTAoUUtXZ1ZH6CN3LK gzPK3DaNuaxX7NLFGovdkAB0cNtg5/JUpGlw4176zx0Znjc4tJBohPBlNn5mWcxAmEfH o1LPVbyu/rJZq7r80YZsxU2Qb24ZkE/2RI2lkcxuYACpb+dYagT++8+paUTQWv/Ie1Cr TIoPc48HuRxzJ2/8BCbTDpKMgMHMCs0/1FJap2a5Hb32zpqWxz5j7wGl5NOZIYRghFo4 dQ== Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by mx0a-0064b401.pphosted.com (PPS) with ESMTPS id 3ppfw6xdaf-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 06 Apr 2023 20:07:28 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FaAk0o1T1ISOxuiG5uJQltyvm6XoO4GcUBQ5VkibouQoJgE32Ux62dlntlxxBoHtXXJlaHnrIfWAAxRporTJbCJAjZ7CNmaHop8bsPvUNhp0rL8q3B/YEU+sdCBuP9aVJsvMlUWQnH3M+rbixgDBlHcQzaGUR+nrvio0YCe7Aq2m47dB227VfDsxSzNKKD6jDHw4lMt13FQXpMm/b7Ntjrxu48+jgCQVcXCaP1AqAeTySiX+gOEQI3POuQcO2dReMIu0IxN1zyPOYVB6AWojUQ2HofgKMIQM7P83nzuXh4k85sBy/IoN/Bi+fbo5JBKgMqXbFXFNEQhc4pGGN7Uesg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sJlbIir+dgUCJS3A7QC3N1oMGwsBMT48YvNL3LIquwU=; b=L9UTub1iBVnmtrWjYK7Uz6yRh2sgpyqVnf5KD32YQ1S1TwCc4tKzhhOqA1vcTn0JMQiVOwBsF6i+MsRZtMcgcHD9Wb6wIJgd/JtBBy3/YVeB28AP6TcAkIoucvkeafHp9dEiR9vJJDM/rOJdHTOfxtRxvP9Xiss2vVrL9XUpP/igcG4FARGPY/oi6v1SR3W3dBZiP1jN6GpAhTFayZ5xVKibox8r7vPjHs119NYUzO9A5zDPF9/e+CuA/mnl860nzVNnOUxfFqFcbqhfphp9HUppSsnWPJbbAc0dsLkhasx+PQDQ0ghkjNxqmnNoJuTgxDF17MFqk7NAY+HZIxCfwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Received: from CO6PR11MB5602.namprd11.prod.outlook.com (2603:10b6:303:13a::5) by DS0PR11MB7829.namprd11.prod.outlook.com (2603:10b6:8:f1::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.33; Fri, 7 Apr 2023 03:07:27 +0000 Received: from CO6PR11MB5602.namprd11.prod.outlook.com ([fe80::ab37:4985:7b95:2b95]) by CO6PR11MB5602.namprd11.prod.outlook.com ([fe80::ab37:4985:7b95:2b95%6]) with mapi id 15.20.6254.035; Fri, 7 Apr 2023 03:07:26 +0000 From: Chen Qi To: bitbake-devel@lists.openembedded.org Subject: [bitbake-devel][PATCH 2/2] runqueue.py: use 'full' line for PSI check Date: Thu, 6 Apr 2023 20:07:15 -0700 Message-Id: <20230407030715.4394-2-Qi.Chen@windriver.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230407030715.4394-1-Qi.Chen@windriver.com> References: <20230407030715.4394-1-Qi.Chen@windriver.com> X-ClientProxiedBy: SJ0PR05CA0095.namprd05.prod.outlook.com (2603:10b6:a03:334::10) To CO6PR11MB5602.namprd11.prod.outlook.com (2603:10b6:303:13a::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR11MB5602:EE_|DS0PR11MB7829:EE_ X-MS-Office365-Filtering-Correlation-Id: 26da423c-805d-40e8-25c8-08db3715370a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pgHuZXWSLrVfNdujIuwrO0hGdhOmudTKPA3psgjWXtH3qV4QkmJUQJeYEir3UHP6mZHgfbTjsGHVytXcKO1Kqjx58mBly/2Tk4QBAygBP2kHGE7qmoJTb61EBYXyMrgY+3AZu62G69emVNcjxRaoJlwhJj2iNMBFxtl55ZFBYTBFPj3o6j9gBVDaf5m3U4CksVUXFWHtBGoTS4fwvdGYEJdLcGaVO2PuFcVqHrSVFXVhGYEesXDcZ1o1ILTAkkprXow1hxrKOUiI1ypPjSED5LY4vKHNLFJN6SH9te18AUlVwlwyVOO69hqWBI7LKiccwgDAFxxmw10coR7i7L6EoCgUe20JlKbDzWoxt8M7r1u0N3Q5skxR6fGbMFIEcOwbUnkX3yAQ8BjV63rb0VpXbRk954EisQCdlEh+cqMdFVkC6A1b043DD7i0ARkdbX6JPTuYtoUL8Sx+ULn31xTlLp+oUYAULc8oElD0q8V2Bx1NPn3kCHRADMVoji7Wb9HFfzBKddXavFVk0Aknx3BoQTdDvsLxUK+CWsf1yBf3rLTi1zXWokeBQ0HX3sK8huD9SmVx+EDiwxxlKVJsXfW32N9o/IdrN1zNBXXB1dVumcg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR11MB5602.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(39850400004)(136003)(396003)(376002)(346002)(366004)(451199021)(2906002)(86362001)(36756003)(966005)(2616005)(186003)(6486002)(6512007)(52116002)(1076003)(6506007)(83380400001)(26005)(6666004)(316002)(66946007)(38100700002)(66556008)(478600001)(66476007)(6916009)(8676002)(41300700001)(5660300002)(38350700002)(8936002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?H0Xy6CiDIyTaQvoczX/nX92Wyg5E?= =?utf-8?q?N1qFSbwfmGGcIGucslMLJFbo6DpH1jTsLgRxiwclqfzpXSJuu+GhK8FuTEH9Z+T1y?= =?utf-8?q?XYqnkTFOxUUSprIqmnWtRXdor9gMkc1H8GOVokBppZThCqk5MrSZY3Kj1Jba7Nh9Q?= =?utf-8?q?7/RYesL3t+JBp7txbgkD3ykoiA9o8Q5WnDt8GvRTrwrwssCMDfhIuTGbOyYhkulkc?= =?utf-8?q?2NJRfFfBG/DpP0wLaBa9CNEQtlY+y7iw4dvyo2wOW/jAOa8g/IwKg8//ViU87lKyf?= =?utf-8?q?tVGjtM261jSnHWT0wdZHoPpJlRiRTW8NAmjZCAa7gsGaHzMO9mMZuBJljgT2z+7sE?= =?utf-8?q?UUEIXi4q7l76ypdo/ZG7RgmnAHaluq8CzlCPh9qwgnduVBf5OCm3pQ13un6hI7frd?= =?utf-8?q?+1V2WDRDNS7Ldu4Zdo3bPBaBMc8Rd8DXhsc7iZUBTp0601fF4d8hP0FamO+gDMFuV?= =?utf-8?q?5z53qs2VumCK+D5ZB33qb+6ecBpCXzAwzkxspcO+l7xVtoXcwcWTQ67jTXeO02Hap?= =?utf-8?q?aXtfW4HQ00uls5M0qvR6AisZGt5tacc9YcwGGxluYV/3qD2aj2160jN6Qjk64JWhL?= =?utf-8?q?J929omhvdupZ7rD3hGfJ9krcJOsPP2FP/igjWCbSsudWXs6aU9Y+U87OO/kAsI1oO?= =?utf-8?q?SfbBYamq0r9SnNcepGLetROz1vHH5GkxeU20CUo8TbtQECmgmO3Xx4WqPidq5W1OZ?= =?utf-8?q?hEJ1fHRzuBY6uUwHW9DQBRMordGzjlvLr3Vza5tduCTI3YN7/IGHD8OjqOhVoxmX9?= =?utf-8?q?/w2E1VO07KgYpO+GLN6ZwaUGwgYkn/E9JPZJkdf9IJSXxXefxTOZf8ll/opsIyGfN?= =?utf-8?q?56iy0IIy0TdnWJNIfmcFHUm2lYOJqN+ngU3ngW5LapXmmnqHYjX5LYc3YLqvyUVst?= =?utf-8?q?xDv+3JSm1ABqVPmvAG/FBqxmkUSiYeZ80/zcHD2u3geYGWz/JMR6JHyoGXMlPQyZ3?= =?utf-8?q?NNGq5fUAeadEsB3FA4LWV6KE/MS2Rzvd59TIIQ44Dpg/Jk4s+TQJ8uXOCoH+5WDlo?= =?utf-8?q?cqIoVLo/h+5qyXVg9uKhWsqr4bcDJecsjPfFGqiyqm41JI7IXRXG6LQ8iFvzcL6Vo?= =?utf-8?q?ndIk0o5WTtLH1nSATLmItZPHvmZeZgc2tUuTAXo5sE0nop2ahn6t0bZmak1aK2+xQ?= =?utf-8?q?431jFOb/ntxpiu1AD03XqDccC49TP3kvGKV1Bip5h8AT6UMJzOivY5Zz1BS5Noh71?= =?utf-8?q?Bhvd2VevIfgosihq9ar98laSNckiCOP4VoGLkhvlrLw+Ly+LEOksUsYfcyV+Yy2j/?= =?utf-8?q?cMQnd+Dd5BSPAwNIRViEheZVOHQAhwIEuOp1Nx1B+nDuxI8uExwR3i/UmeaItx4F3?= =?utf-8?q?8c6ln9Tr07LObAVv+vWUAojZy1ZrWgU0OB0u61lmkauN2PLFhoP7+nAvhd7zAxRcE?= =?utf-8?q?I1/g8RN2R9P7iv7FpcExT4tnh84kXpIG8Z4fJJ8MAq/yl2Eztbw5fhQvwZvJA4Oot?= =?utf-8?q?vKPZk2+Zau9kaXQTzRUAQs83RRBZwyk/dAyXaFoi4mVVjiFiMEodq8/8yxcwEZ0/G?= =?utf-8?q?2Jpz+9kJ2W8m3ol/OgibkS9Lw8O5lQfPWg=3D=3D?= X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 26da423c-805d-40e8-25c8-08db3715370a X-MS-Exchange-CrossTenant-AuthSource: CO6PR11MB5602.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2023 03:07:26.1237 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /MmNL8hFOCHmtAthRsWewSeR4nYqHtAtluwLUwEcNB8wap1kq9fx41p2DshmH0+aEQVlPbA8sEyjE/jlkzIO2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7829 X-Proofpoint-ORIG-GUID: orLjEQ2xt-9NrVQfDYClZ4gYRKG5SghD X-Proofpoint-GUID: orLjEQ2xt-9NrVQfDYClZ4gYRKG5SghD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-06_14,2023-04-06_03,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=483 lowpriorityscore=0 impostorscore=0 adultscore=0 phishscore=0 mlxscore=0 priorityscore=1501 malwarescore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2304070028 X-MIME-Autoconverted: from 8bit to quoted-printable by mx0a-0064b401.pphosted.com id 3372iTZj004635 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 07 Apr 2023 03:07:38 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14683 According to kernel PSI doc[1], for /proc/pressure/* interface files, the first line is the 'some' line and the second line is the 'full' line. Quoting from the doc: """ The "some" line indicates the share of time in which at least some tasks are stalled on a given resource. The "full" line indicates the share of time in which all non-idle tasks are stalled on a given resource simultaneously. In this state actual CPU cycles are going to waste, and a workload that spends extended time in this state is considered to be thrashing. This has severe impact on performance, and it’s useful to distinguish this situation from a state where some tasks are stalled but the CPU is still doing productive work. """ We can see that the 'full' line is a better measurement to check if things are slowed down as a whole. Also, the /proc/pressure/cpu's 'full' line may not be available on some systems, so fall back to the 'some' line. [1] https://www.kernel.org/doc/html/latest/accounting/psi.html Signed-off-by: Chen Qi --- bitbake/lib/bb/runqueue.py | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py index 02f1474540..e3198dcdeb 100644 --- a/bitbake/lib/bb/runqueue.py +++ b/bitbake/lib/bb/runqueue.py @@ -173,10 +173,13 @@ class RunQueueScheduler(object): with open("/proc/pressure/cpu") as cpu_pressure_fds, \ open("/proc/pressure/io") as io_pressure_fds, \ open("/proc/pressure/memory") as memory_pressure_fds: - - self.prev_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1] - self.prev_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1] - self.prev_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1] + cpu_lines = cpu_pressure_fds.readlines() + if len(cpu_lines) == 1: + self.prev_cpu_pressure = cpu_lines[0].split()[4].split("=")[1] + else: + self.prev_cpu_pressure = cpu_lines[1].split()[4].split("=")[1] + self.prev_io_pressure = io_pressure_fds.readlines()[1].split()[4].split("=")[1] + self.prev_memory_pressure = memory_pressure_fds.readlines()[1].split()[4].split("=")[1] self.prev_pressure_time = time.time() self.check_pressure = True except: @@ -194,10 +197,14 @@ class RunQueueScheduler(object): with open("/proc/pressure/cpu") as cpu_pressure_fds, \ open("/proc/pressure/io") as io_pressure_fds, \ open("/proc/pressure/memory") as memory_pressure_fds: - # extract "total" from /proc/pressure/{cpu|io} - curr_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1] - curr_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1] - curr_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1] + # extract "total" from /proc/pressure/{cpu|io|memory} + cpu_lines = cpu_pressure_fds.readlines() + if len(cpu_lines) == 1: + curr_cpu_pressure = cpu_lines[0].split()[4].split("=")[1] + else: + curr_cpu_pressure = cpu_lines[1].split()[4].split("=")[1] + curr_io_pressure = io_pressure_fds.readlines()[1].split()[4].split("=")[1] + curr_memory_pressure = memory_pressure_fds.readlines()[1].split()[4].split("=")[1] now = time.time() tdiff = now - self.prev_pressure_time if tdiff > 1.0: