Documentation/admin-guide/pm/amd-pstate.rst

   1 .. SPDX-License-Identifier: GPL-2.0
   2 .. include:: <isonum.txt>
   3
   4 ===============================================
   5 ``amd-pstate`` CPU Performance Scaling Driver
   6 ===============================================
   7
   8 :Copyright: |copy| 2021 Advanced Micro Devices, Inc.
   9
  10 :Author: Huang Rui <ray.huang@amd.com>
  11
  12
  13 Introduction
  14 ===================
  15
  16 ``amd-pstate`` is the AMD CPU performance scaling driver that introduces a
  17 new CPU frequency control mechanism on modern AMD APU and CPU series in
  18 Linux kernel. The new mechanism is based on Collaborative Processor
  19 Performance Control (CPPC) which provides finer grain frequency management
  20 than legacy ACPI hardware P-States. Current AMD CPU/APU platforms are using
  21 the ACPI P-states driver to manage CPU frequency and clocks with switching
  22 only in 3 P-states. CPPC replaces the ACPI P-states controls and allows a
  23 flexible, low-latency interface for the Linux kernel to directly
  24 communicate the performance hints to hardware.
  25
  26 ``amd-pstate`` leverages the Linux kernel governors such as ``schedutil``,
  27 ``ondemand``, etc. to manage the performance hints which are provided by
  28 CPPC hardware functionality that internally follows the hardware
  29 specification (for details refer to AMD64 Architecture Programmer's Manual
  30 Volume 2: System Programming [1]_). Currently, ``amd-pstate`` supports basic
  31 frequency control function according to kernel governors on some of the
  32 Zen2 and Zen3 processors, and we will implement more AMD specific functions
  33 in future after we verify them on the hardware and SBIOS.
  34
  35
  36 AMD CPPC Overview
  37 =======================
  38
  39 Collaborative Processor Performance Control (CPPC) interface enumerates a
  40 continuous, abstract, and unit-less performance value in a scale that is
  41 not tied to a specific performance state / frequency. This is an ACPI
  42 standard [2]_ which software can specify application performance goals and
  43 hints as a relative target to the infrastructure limits. AMD processors
  44 provide the low latency register model (MSR) instead of an AML code
  45 interpreter for performance adjustments. ``amd-pstate`` will initialize a
  46 ``struct cpufreq_driver`` instance, ``amd_pstate_driver``, with the callbacks
  47 to manage each performance update behavior. ::
  48
  49  Highest Perf ------>+-----------------------+                         +-----------------------+
  50                      |                       |                         |                       |
  51                      |                       |                         |                       |
  52                      |                       |          Max Perf  ---->|                       |
  53                      |                       |                         |                       |
  54                      |                       |                         |                       |
  55  Nominal Perf ------>+-----------------------+                         +-----------------------+
  56                      |                       |                         |                       |
  57                      |                       |                         |                       |
  58                      |                       |                         |                       |
  59                      |                       |                         |                       |
  60                      |                       |                         |                       |
  61                      |                       |                         |                       |
  62                      |                       |      Desired Perf  ---->|                       |
  63                      |                       |                         |                       |
  64                      |                       |                         |                       |
  65                      |                       |                         |                       |
  66                      |                       |                         |                       |
  67                      |                       |                         |                       |
  68                      |                       |                         |                       |
  69                      |                       |                         |                       |
  70                      |                       |                         |                       |
  71                      |                       |                         |                       |
  72   Lowest non-        |                       |                         |                       |
  73   linear perf ------>+-----------------------+                         +-----------------------+
  74                      |                       |                         |                       |
  75                      |                       |       Lowest perf  ---->|                       |
  76                      |                       |                         |                       |
  77   Lowest perf ------>+-----------------------+                         +-----------------------+
  78                      |                       |                         |                       |
  79                      |                       |                         |                       |
  80                      |                       |                         |                       |
  81           0   ------>+-----------------------+                         +-----------------------+
  82
  83                                      AMD P-States Performance Scale
  84
  85
  86 .. _perf_cap:
  87
  88 AMD CPPC Performance Capability
  89 --------------------------------
  90
  91 Highest Performance (RO)
  92 .........................
  93
  94 This is the absolute maximum performance an individual processor may reach,
  95 assuming ideal conditions. This performance level may not be sustainable
  96 for long durations and may only be achievable if other platform components
  97 are in a specific state; for example, it may require other processors to be in
  98 an idle state. This would be equivalent to the highest frequencies
  99 supported by the processor.
 100
 101 Nominal (Guaranteed) Performance (RO)
 102 ......................................
 103
 104 This is the maximum sustained performance level of the processor, assuming
 105 ideal operating conditions. In the absence of an external constraint (power,
 106 thermal, etc.), this is the performance level the processor is expected to
 107 be able to maintain continuously. All cores/processors are expected to be
 108 able to sustain their nominal performance state simultaneously.
 109
 110 Lowest non-linear Performance (RO)
 111 ...................................
 112
 113 This is the lowest performance level at which nonlinear power savings are
 114 achieved, for example, due to the combined effects of voltage and frequency
 115 scaling. Above this threshold, lower performance levels should be generally
 116 more energy efficient than higher performance levels. This register
 117 effectively conveys the most efficient performance level to ``amd-pstate``.
 118
 119 Lowest Performance (RO)
 120 ........................
 121
 122 This is the absolute lowest performance level of the processor. Selecting a
 123 performance level lower than the lowest nonlinear performance level may
 124 cause an efficiency penalty but should reduce the instantaneous power
 125 consumption of the processor.
 126
 127 AMD CPPC Performance Control
 128 ------------------------------
 129
 130 ``amd-pstate`` passes performance goals through these registers. The
 131 register drives the behavior of the desired performance target.
 132
 133 Minimum requested performance (RW)
 134 ...................................
 135
 136 ``amd-pstate`` specifies the minimum allowed performance level.
 137
 138 Maximum requested performance (RW)
 139 ...................................
 140
 141 ``amd-pstate`` specifies a limit the maximum performance that is expected
 142 to be supplied by the hardware.
 143
 144 Desired performance target (RW)
 145 ...................................
 146
 147 ``amd-pstate`` specifies a desired target in the CPPC performance scale as
 148 a relative number. This can be expressed as percentage of nominal
 149 performance (infrastructure max). Below the nominal sustained performance
 150 level, desired performance expresses the average performance level of the
 151 processor subject to hardware. Above the nominal performance level,
 152 the processor must provide at least nominal performance requested and go higher
 153 if current operating conditions allow.
 154
 155 Energy Performance Preference (EPP) (RW)
 156 .........................................
 157
 158 This attribute provides a hint to the hardware if software wants to bias
 159 toward performance (0x0) or energy efficiency (0xff).
 160
 161
 162 Key Governors Support
 163 =======================
 164
 165 ``amd-pstate`` can be used with all the (generic) scaling governors listed
 166 by the ``scaling_available_governors`` policy attribute in ``sysfs``. Then,
 167 it is responsible for the configuration of policy objects corresponding to
 168 CPUs and provides the ``CPUFreq`` core (and the scaling governors attached
 169 to the policy objects) with accurate information on the maximum and minimum
 170 operating frequencies supported by the hardware. Users can check the
 171 ``scaling_cur_freq`` information comes from the ``CPUFreq`` core.
 172
 173 ``amd-pstate`` mainly supports ``schedutil`` and ``ondemand`` for dynamic
 174 frequency control. It is to fine tune the processor configuration on
 175 ``amd-pstate`` to the ``schedutil`` with CPU CFS scheduler. ``amd-pstate``
 176 registers the adjust_perf callback to implement performance update behavior
 177 similar to CPPC. It is initialized by ``sugov_start`` and then populates the
 178 CPU's update_util_data pointer to assign ``sugov_update_single_perf`` as the
 179 utilization update callback function in the CPU scheduler. The CPU scheduler
 180 will call ``cpufreq_update_util`` and assigns the target performance according
 181 to the ``struct sugov_cpu`` that the utilization update belongs to.
 182 Then, ``amd-pstate`` updates the desired performance according to the CPU
 183 scheduler assigned.
 184
 185 .. _processor_support:
 186
 187 Processor Support
 188 =======================
 189
 190 The ``amd-pstate`` initialization will fail if the ``_CPC`` entry in the ACPI
 191 SBIOS does not exist in the detected processor. It uses ``acpi_cpc_valid``
 192 to check the existence of ``_CPC``. All Zen based processors support the legacy
 193 ACPI hardware P-States function, so when ``amd-pstate`` fails initialization,
 194 the kernel will fall back to initialize the ``acpi-cpufreq`` driver.
 195
 196 There are two types of hardware implementations for ``amd-pstate``: one is
 197 `Full MSR Support <perf_cap_>`_ and another is `Shared Memory Support
 198 <perf_cap_>`_. It can use the :c:macro:`X86_FEATURE_CPPC` feature flag to
 199 indicate the different types. (For details, refer to the Processor Programming
 200 Reference (PPR) for AMD Family 19h Model 51h, Revision A1 Processors [3]_.)
 201 ``amd-pstate`` is to register different ``static_call`` instances for different
 202 hardware implementations.
 203
 204 Currently, some of the Zen2 and Zen3 processors support ``amd-pstate``. In the
 205 future, it will be supported on more and more AMD processors.
 206
 207 Full MSR Support
 208 -----------------
 209
 210 Some new Zen3 processors such as Cezanne provide the MSR registers directly
 211 while the :c:macro:`X86_FEATURE_CPPC` CPU feature flag is set.
 212 ``amd-pstate`` can handle the MSR register to implement the fast switch
 213 function in ``CPUFreq`` that can reduce the latency of frequency control in
 214 interrupt context. The functions with a ``pstate_xxx`` prefix represent the
 215 operations on MSR registers.
 216
 217 Shared Memory Support
 218 ----------------------
 219
 220 If the :c:macro:`X86_FEATURE_CPPC` CPU feature flag is not set, the
 221 processor supports the shared memory solution. In this case, ``amd-pstate``
 222 uses the ``cppc_acpi`` helper methods to implement the callback functions
 223 that are defined on ``static_call``. The functions with the ``cppc_xxx`` prefix
 224 represent the operations of ACPI CPPC helpers for the shared memory solution.
 225
 226
 227 AMD P-States and ACPI hardware P-States always can be supported in one
 228 processor. But AMD P-States has the higher priority and if it is enabled
 229 with :c:macro:`MSR_AMD_CPPC_ENABLE` or ``cppc_set_enable``, it will respond
 230 to the request from AMD P-States.
 231
 232
 233 User Space Interface in ``sysfs`` - Per-policy control
 234 ======================================================
 235
 236 ``amd-pstate`` exposes several global attributes (files) in ``sysfs`` to
 237 control its functionality at the system level. They are located in the
 238 ``/sys/devices/system/cpu/cpufreq/policyX/`` directory and affect all CPUs. ::
 239
 240  root@hr-test1:/home/ray# ls /sys/devices/system/cpu/cpufreq/policy0/*amd*
 241  /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_highest_perf
 242  /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_lowest_nonlinear_freq
 243  /sys/devices/system/cpu/cpufreq/policy0/amd_pstate_max_freq
 244
 245
 246 ``amd_pstate_highest_perf / amd_pstate_max_freq``
 247
 248 Maximum CPPC performance and CPU frequency that the driver is allowed to
 249 set, in percent of the maximum supported CPPC performance level (the highest
 250 performance supported in `AMD CPPC Performance Capability <perf_cap_>`_).
 251 In some ASICs, the highest CPPC performance is not the one in the ``_CPC``
 252 table, so we need to expose it to sysfs. If boost is not active, but
 253 still supported, this maximum frequency will be larger than the one in
 254 ``cpuinfo``.
 255 This attribute is read-only.
 256
 257 ``amd_pstate_lowest_nonlinear_freq``
 258
 259 The lowest non-linear CPPC CPU frequency that the driver is allowed to set,
 260 in percent of the maximum supported CPPC performance level. (Please see the
 261 lowest non-linear performance in `AMD CPPC Performance Capability
 262 <perf_cap_>`_.)
 263 This attribute is read-only.
 264
 265 ``energy_performance_available_preferences``
 266
 267 A list of all the supported EPP preferences that could be used for
 268 ``energy_performance_preference`` on this system.
 269 These profiles represent different hints that are provided
 270 to the low-level firmware about the user's desired energy vs efficiency
 271 tradeoff.  ``default`` represents the epp value is set by platform
 272 firmware. This attribute is read-only.
 273
 274 ``energy_performance_preference``
 275
 276 The current energy performance preference can be read from this attribute.
 277 and user can change current preference according to energy or performance needs
 278 Please get all support profiles list from
 279 ``energy_performance_available_preferences`` attribute, all the profiles are
 280 integer values defined between 0 to 255 when EPP feature is enabled by platform
 281 firmware, if EPP feature is disabled, driver will ignore the written value
 282 This attribute is read-write.
 283
 284 Other performance and frequency values can be read back from
 285 ``/sys/devices/system/cpu/cpuX/acpi_cppc/``, see :ref:`cppc_sysfs`.
 286
 287
 288 ``amd-pstate`` vs ``acpi-cpufreq``
 289 ======================================
 290
 291 On the majority of AMD platforms supported by ``acpi-cpufreq``, the ACPI tables
 292 provided by the platform firmware are used for CPU performance scaling, but
 293 only provide 3 P-states on AMD processors.
 294 However, on modern AMD APU and CPU series, hardware provides the Collaborative
 295 Processor Performance Control according to the ACPI protocol and customizes this
 296 for AMD platforms. That is, fine-grained and continuous frequency ranges
 297 instead of the legacy hardware P-states. ``amd-pstate`` is the kernel
 298 module which supports the new AMD P-States mechanism on most of the future AMD
 299 platforms. The AMD P-States mechanism is the more performance and energy
 300 efficiency frequency management method on AMD processors.
 301
 302
 303 AMD Pstate Driver Operation Modes
 304 =================================
 305
 306 ``amd_pstate`` CPPC has 3 operation modes: autonomous (active) mode,
 307 non-autonomous (passive) mode and guided autonomous (guided) mode.
 308 Active/passive/guided mode can be chosen by different kernel parameters.
 309
 310 - In autonomous mode, platform ignores the desired performance level request
 311   and takes into account only the values set to the minimum, maximum and energy
 312   performance preference registers.
 313 - In non-autonomous mode, platform gets desired performance level
 314   from OS directly through Desired Performance Register.
 315 - In guided-autonomous mode, platform sets operating performance level
 316   autonomously according to the current workload and within the limits set by
 317   OS through min and max performance registers.
 318
 319 Active Mode
 320 ------------
 321
 322 ``amd_pstate=active``
 323
 324 This is the low-level firmware control mode which is implemented by ``amd_pstate_epp``
 325 driver with ``amd_pstate=active`` passed to the kernel in the command line.
 326 In this mode, ``amd_pstate_epp`` driver provides a hint to the hardware if software
 327 wants to bias toward performance (0x0) or energy efficiency (0xff) to the CPPC firmware.
 328 then CPPC power algorithm will calculate the runtime workload and adjust the realtime
 329 cores frequency according to the power supply and thermal, core voltage and some other
 330 hardware conditions.
 331
 332 Passive Mode
 333 ------------
 334
 335 ``amd_pstate=passive``
 336
 337 It will be enabled if the ``amd_pstate=passive`` is passed to the kernel in the command line.
 338 In this mode, ``amd_pstate`` driver software specifies a desired QoS target in the CPPC
 339 performance scale as a relative number. This can be expressed as percentage of nominal
 340 performance (infrastructure max). Below the nominal sustained performance level,
 341 desired performance expresses the average performance level of the processor subject
 342 to the Performance Reduction Tolerance register. Above the nominal performance level,
 343 processor must provide at least nominal performance requested and go higher if current
 344 operating conditions allow.
 345
 346 Guided Mode
 347 -----------
 348
 349 ``amd_pstate=guided``
 350
 351 If ``amd_pstate=guided`` is passed to kernel command line option then this mode
 352 is activated.  In this mode, driver requests minimum and maximum performance
 353 level and the platform autonomously selects a performance level in this range
 354 and appropriate to the current workload.
 355
 356 User Space Interface in ``sysfs`` - General
 357 ===========================================
 358
 359 Global Attributes
 360 -----------------
 361
 362 ``amd-pstate`` exposes several global attributes (files) in ``sysfs`` to
 363 control its functionality at the system level.  They are located in the
 364 ``/sys/devices/system/cpu/amd_pstate/`` directory and affect all CPUs.
 365
 366 ``status``
 367         Operation mode of the driver: "active", "passive" or "disable".
 368
 369         "active"
 370                 The driver is functional and in the ``active mode``
 371
 372         "passive"
 373                 The driver is functional and in the ``passive mode``
 374
 375         "guided"
 376                 The driver is functional and in the ``guided mode``
 377
 378         "disable"
 379                 The driver is unregistered and not functional now.
 380
 381         This attribute can be written to in order to change the driver's
 382         operation mode or to unregister it.  The string written to it must be
 383         one of the possible values of it and, if successful, writing one of
 384         these values to the sysfs file will cause the driver to switch over
 385         to the operation mode represented by that string - or to be
 386         unregistered in the "disable" case.
 387
 388 ``cpupower`` tool support for ``amd-pstate``
 389 ===============================================
 390
 391 ``amd-pstate`` is supported by the ``cpupower`` tool, which can be used to dump
 392 frequency information. Development is in progress to support more and more
 393 operations for the new ``amd-pstate`` module with this tool. ::
 394
 395  root@hr-test1:/home/ray# cpupower frequency-info
 396  analyzing CPU 0:
 397    driver: amd-pstate
 398    CPUs which run at the same hardware frequency: 0
 399    CPUs which need to have their frequency coordinated by software: 0
 400    maximum transition latency: 131 us
 401    hardware limits: 400 MHz - 4.68 GHz
 402    available cpufreq governors: ondemand conservative powersave userspace performance schedutil
 403    current policy: frequency should be within 400 MHz and 4.68 GHz.
 404                    The governor "schedutil" may decide which speed to use
 405                    within this range.
 406    current CPU frequency: Unable to call hardware
 407    current CPU frequency: 4.02 GHz (asserted by call to kernel)
 408    boost state support:
 409      Supported: yes
 410      Active: yes
 411      AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.68 GHz.
 412      AMD PSTATE Nominal Performance: 117. Nominal Frequency: 3.30 GHz.
 413      AMD PSTATE Lowest Non-linear Performance: 39. Lowest Non-linear Frequency: 1.10 GHz.
 414      AMD PSTATE Lowest Performance: 15. Lowest Frequency: 400 MHz.
 415
 416
 417 Diagnostics and Tuning
 418 =======================
 419
 420 Trace Events
 421 --------------
 422
 423 There are two static trace events that can be used for ``amd-pstate``
 424 diagnostics. One of them is the ``cpu_frequency`` trace event generally used
 425 by ``CPUFreq``, and the other one is the ``amd_pstate_perf`` trace event
 426 specific to ``amd-pstate``.  The following sequence of shell commands can
 427 be used to enable them and see their output (if the kernel is
 428 configured to support event tracing). ::
 429
 430  root@hr-test1:/home/ray# cd /sys/kernel/tracing/
 431  root@hr-test1:/sys/kernel/tracing# echo 1 > events/amd_cpu/enable
 432  root@hr-test1:/sys/kernel/tracing# cat trace
 433  # tracer: nop
 434  #
 435  # entries-in-buffer/entries-written: 47827/42233061   #P:2
 436  #
 437  #                                _-----=> irqs-off
 438  #                               / _----=> need-resched
 439  #                              | / _---=> hardirq/softirq
 440  #                              || / _--=> preempt-depth
 441  #                              ||| /     delay
 442  #           TASK-PID     CPU#  ||||   TIMESTAMP  FUNCTION
 443  #              | |         |   ||||      |         |
 444           <idle>-0       [015] dN...  4995.979886: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=15 changed=false fast_switch=true
 445           <idle>-0       [007] d.h..  4995.979893: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=7 changed=false fast_switch=true
 446              cat-2161    [000] d....  4995.980841: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=0 changed=false fast_switch=true
 447             sshd-2125    [004] d.s..  4995.980968: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=4 changed=false fast_switch=true
 448           <idle>-0       [007] d.s..  4995.980968: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=7 changed=false fast_switch=true
 449           <idle>-0       [003] d.s..  4995.980971: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=3 changed=false fast_switch=true
 450           <idle>-0       [011] d.s..  4995.980996: amd_pstate_perf: amd_min_perf=85 amd_des_perf=85 amd_max_perf=166 cpu_id=11 changed=false fast_switch=true
 451
 452 The ``cpu_frequency`` trace event will be triggered either by the ``schedutil`` scaling
 453 governor (for the policies it is attached to), or by the ``CPUFreq`` core (for the
 454 policies with other scaling governors).
 455
 456
 457 Tracer Tool
 458 -------------
 459
 460 ``amd_pstate_tracer.py`` can record and parse ``amd-pstate`` trace log, then
 461 generate performance plots. This utility can be used to debug and tune the
 462 performance of ``amd-pstate`` driver. The tracer tool needs to import intel
 463 pstate tracer.
 464
 465 Tracer tool located in ``linux/tools/power/x86/amd_pstate_tracer``. It can be
 466 used in two ways. If trace file is available, then directly parse the file
 467 with command ::
 468
 469  ./amd_pstate_trace.py [-c cpus] -t <trace_file> -n <test_name>
 470
 471 Or generate trace file with root privilege, then parse and plot with command ::
 472
 473  sudo ./amd_pstate_trace.py [-c cpus] -n <test_name> -i <interval> [-m kbytes]
 474
 475 The test result can be found in ``results/test_name``. Following is the example
 476 about part of the output. ::
 477
 478  common_cpu  common_secs  common_usecs  min_perf  des_perf  max_perf  freq    mperf   apef    tsc       load   duration_ms  sample_num  elapsed_time  common_comm
 479  CPU_005     712          116384        39        49        166       0.7565  9645075 2214891 38431470  25.1   11.646       469         2.496         kworker/5:0-40
 480  CPU_006     712          116408        39        49        166       0.6769  8950227 1839034 37192089  24.06  11.272       470         2.496         kworker/6:0-1264
 481
 482 Unit Tests for amd-pstate
 483 -------------------------
 484
 485 ``amd-pstate-ut`` is a test module for testing the ``amd-pstate`` driver.
 486
 487  * It can help all users to verify their processor support (SBIOS/Firmware or Hardware).
 488
 489  * Kernel can have a basic function test to avoid the kernel regression during the update.
 490
 491  * We can introduce more functional or performance tests to align the result together, it will benefit power and performance scale optimization.
 492
 493 1. Test case descriptions
 494
 495     1). Basic tests
 496
 497         Test prerequisite and basic functions for the ``amd-pstate`` driver.
 498
 499         +---------+--------------------------------+------------------------------------------------------------------------------------+
 500         | Index   | Functions                      | Description                                                                        |
 501         +=========+================================+====================================================================================+
 502         | 1       | amd_pstate_ut_acpi_cpc_valid   || Check whether the _CPC object is present in SBIOS.                                |
 503         |         |                                ||                                                                                   |
 504         |         |                                || The detail refer to `Processor Support <processor_support_>`_.                    |
 505         +---------+--------------------------------+------------------------------------------------------------------------------------+
 506         | 2       | amd_pstate_ut_check_enabled    || Check whether AMD P-State is enabled.                                             |
 507         |         |                                ||                                                                                   |
 508         |         |                                || AMD P-States and ACPI hardware P-States always can be supported in one processor. |
 509         |         |                                | But AMD P-States has the higher priority and if it is enabled with                 |
 510         |         |                                | :c:macro:`MSR_AMD_CPPC_ENABLE` or ``cppc_set_enable``, it will respond to the      |
 511         |         |                                | request from AMD P-States.                                                         |
 512         +---------+--------------------------------+------------------------------------------------------------------------------------+
 513         | 3       | amd_pstate_ut_check_perf       || Check if the each performance values are reasonable.                              |
 514         |         |                                || highest_perf >= nominal_perf > lowest_nonlinear_perf > lowest_perf > 0.           |
 515         +---------+--------------------------------+------------------------------------------------------------------------------------+
 516         | 4       | amd_pstate_ut_check_freq       || Check if the each frequency values and max freq when set support boost mode       |
 517         |         |                                | are reasonable.                                                                    |
 518         |         |                                || max_freq >= nominal_freq > lowest_nonlinear_freq > min_freq > 0                   |
 519         |         |                                || If boost is not active but supported, this maximum frequency will be larger than  |
 520         |         |                                | the one in ``cpuinfo``.                                                            |
 521         +---------+--------------------------------+------------------------------------------------------------------------------------+
 522
 523     2). Tbench test
 524
 525         Test and monitor the cpu changes when running tbench benchmark under the specified governor.
 526         These changes include desire performance, frequency, load, performance, energy etc.
 527         The specified governor is ondemand or schedutil.
 528         Tbench can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
 529
 530     3). Gitsource test
 531
 532         Test and monitor the cpu changes when running gitsource benchmark under the specified governor.
 533         These changes include desire performance, frequency, load, time, energy etc.
 534         The specified governor is ondemand or schedutil.
 535         Gitsource can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
 536
 537 #. How to execute the tests
 538
 539    We use test module in the kselftest frameworks to implement it.
 540    We create ``amd-pstate-ut`` module and tie it into kselftest.(for
 541    details refer to Linux Kernel Selftests [4]_).
 542
 543     1). Build
 544
 545         + open the :c:macro:`CONFIG_X86_AMD_PSTATE` configuration option.
 546         + set the :c:macro:`CONFIG_X86_AMD_PSTATE_UT` configuration option to M.
 547         + make project
 548         + make selftest ::
 549
 550             $ cd linux
 551             $ make -C tools/testing/selftests
 552
 553         + make perf ::
 554
 555             $ cd tools/perf/
 556             $ make
 557
 558
 559     2). Installation & Steps ::
 560
 561         $ make -C tools/testing/selftests install INSTALL_PATH=~/kselftest
 562         $ cp tools/perf/perf /usr/bin/perf
 563         $ sudo ./kselftest/run_kselftest.sh -c amd-pstate
 564
 565     3). Specified test case ::
 566
 567         $ cd ~/kselftest/amd-pstate
 568         $ sudo ./run.sh -t basic
 569         $ sudo ./run.sh -t tbench
 570         $ sudo ./run.sh -t tbench -m acpi-cpufreq
 571         $ sudo ./run.sh -t gitsource
 572         $ sudo ./run.sh -t gitsource -m acpi-cpufreq
 573         $ ./run.sh --help
 574         ./run.sh: illegal option -- -
 575         Usage: ./run.sh [OPTION...]
 576                 [-h <help>]
 577                 [-o <output-file-for-dump>]
 578                 [-c <all: All testing,
 579                      basic: Basic testing,
 580                      tbench: Tbench testing,
 581                      gitsource: Gitsource testing.>]
 582                 [-t <tbench time limit>]
 583                 [-p <tbench process number>]
 584                 [-l <loop times for tbench>]
 585                 [-i <amd tracer interval>]
 586                 [-m <comparative test: acpi-cpufreq>]
 587
 588
 589     4). Results
 590
 591         + basic
 592
 593          When you finish test, you will get the following log info ::
 594
 595           $ dmesg | grep "amd_pstate_ut" | tee log.txt
 596           [12977.570663] amd_pstate_ut: 1    amd_pstate_ut_acpi_cpc_valid  success!
 597           [12977.570673] amd_pstate_ut: 2    amd_pstate_ut_check_enabled   success!
 598           [12977.571207] amd_pstate_ut: 3    amd_pstate_ut_check_perf      success!
 599           [12977.571212] amd_pstate_ut: 4    amd_pstate_ut_check_freq      success!
 600
 601         + tbench
 602
 603          When you finish test, you will get selftest.tbench.csv and png images.
 604          The selftest.tbench.csv file contains the raw data and the drop of the comparative test.
 605          The png images shows the performance, energy and performan per watt of each test.
 606          Open selftest.tbench.csv :
 607
 608          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 609          + Governor                                        | Round        | Des-perf | Freq    | Load     | Performance | Energy  | Performance Per Watt |
 610          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 611          + Unit                                            |              |          | GHz     |          | MB/s        | J       | MB/J                 |
 612          +=================================================+==============+==========+=========+==========+=============+=========+======================+
 613          + amd-pstate-ondemand                             | 1            |          |         |          | 2504.05     | 1563.67 | 158.5378             |
 614          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 615          + amd-pstate-ondemand                             | 2            |          |         |          | 2243.64     | 1430.32 | 155.2941             |
 616          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 617          + amd-pstate-ondemand                             | 3            |          |         |          | 2183.88     | 1401.32 | 154.2860             |
 618          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 619          + amd-pstate-ondemand                             | Average      |          |         |          | 2310.52     | 1465.1  | 156.1268             |
 620          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 621          + amd-pstate-schedutil                            | 1            | 165.329  | 1.62257 | 99.798   | 2136.54     | 1395.26 | 151.5971             |
 622          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 623          + amd-pstate-schedutil                            | 2            | 166      | 1.49761 | 99.9993  | 2100.56     | 1380.5  | 150.6377             |
 624          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 625          + amd-pstate-schedutil                            | 3            | 166      | 1.47806 | 99.9993  | 2084.12     | 1375.76 | 149.9737             |
 626          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 627          + amd-pstate-schedutil                            | Average      | 165.776  | 1.53275 | 99.9322  | 2107.07     | 1383.84 | 150.7399             |
 628          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 629          + acpi-cpufreq-ondemand                           | 1            |          |         |          | 2529.9      | 1564.4  | 160.0997             |
 630          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 631          + acpi-cpufreq-ondemand                           | 2            |          |         |          | 2249.76     | 1432.97 | 155.4297             |
 632          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 633          + acpi-cpufreq-ondemand                           | 3            |          |         |          | 2181.46     | 1406.88 | 153.5060             |
 634          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 635          + acpi-cpufreq-ondemand                           | Average      |          |         |          | 2320.37     | 1468.08 | 156.4741             |
 636          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 637          + acpi-cpufreq-schedutil                          | 1            |          |         |          | 2137.64     | 1385.24 | 152.7723             |
 638          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 639          + acpi-cpufreq-schedutil                          | 2            |          |         |          | 2107.05     | 1372.23 | 152.0138             |
 640          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 641          + acpi-cpufreq-schedutil                          | 3            |          |         |          | 2085.86     | 1365.35 | 151.2433             |
 642          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 643          + acpi-cpufreq-schedutil                          | Average      |          |         |          | 2110.18     | 1374.27 | 152.0136             |
 644          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 645          + acpi-cpufreq-ondemand VS acpi-cpufreq-schedutil | Comprison(%) |          |         |          | -9.0584     | -6.3899 | -2.8506              |
 646          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 647          + amd-pstate-ondemand VS amd-pstate-schedutil     | Comprison(%) |          |         |          | 8.8053      | -5.5463 | -3.4503              |
 648          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 649          + acpi-cpufreq-ondemand VS amd-pstate-ondemand    | Comprison(%) |          |         |          | -0.4245     | -0.2029 | -0.2219              |
 650          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 651          + acpi-cpufreq-schedutil VS amd-pstate-schedutil  | Comprison(%) |          |         |          | -0.1473     | 0.6963  | -0.8378              |
 652          +-------------------------------------------------+--------------+----------+---------+----------+-------------+---------+----------------------+
 653
 654         + gitsource
 655
 656          When you finish test, you will get selftest.gitsource.csv and png images.
 657          The selftest.gitsource.csv file contains the raw data and the drop of the comparative test.
 658          The png images shows the performance, energy and performan per watt of each test.
 659          Open selftest.gitsource.csv :
 660
 661          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 662          + Governor                                        | Round        | Des-perf | Freq     | Load     | Time        | Energy  | Performance Per Watt |
 663          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 664          + Unit                                            |              |          | GHz      |          | s           | J       | 1/J                  |
 665          +=================================================+==============+==========+==========+==========+=============+=========+======================+
 666          + amd-pstate-ondemand                             | 1            | 50.119   | 2.10509  | 23.3076  | 475.69      | 865.78  | 0.001155027          |
 667          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 668          + amd-pstate-ondemand                             | 2            | 94.8006  | 1.98771  | 56.6533  | 467.1       | 839.67  | 0.001190944          |
 669          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 670          + amd-pstate-ondemand                             | 3            | 76.6091  | 2.53251  | 43.7791  | 467.69      | 855.85  | 0.001168429          |
 671          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 672          + amd-pstate-ondemand                             | Average      | 73.8429  | 2.20844  | 41.2467  | 470.16      | 853.767 | 0.001171279          |
 673          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 674          + amd-pstate-schedutil                            | 1            | 165.919  | 1.62319  | 98.3868  | 464.17      | 866.8   | 0.001153668          |
 675          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 676          + amd-pstate-schedutil                            | 2            | 165.97   | 1.31309  | 99.5712  | 480.15      | 880.4   | 0.001135847          |
 677          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 678          + amd-pstate-schedutil                            | 3            | 165.973  | 1.28448  | 99.9252  | 481.79      | 867.02  | 0.001153375          |
 679          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 680          + amd-pstate-schedutil                            | Average      | 165.954  | 1.40692  | 99.2944  | 475.37      | 871.407 | 0.001147569          |
 681          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 682          + acpi-cpufreq-ondemand                           | 1            |          |          |          | 2379.62     | 742.96  | 0.001345967          |
 683          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 684          + acpi-cpufreq-ondemand                           | 2            |          |          |          | 441.74      | 817.49  | 0.001223256          |
 685          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 686          + acpi-cpufreq-ondemand                           | 3            |          |          |          | 455.48      | 820.01  | 0.001219497          |
 687          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 688          + acpi-cpufreq-ondemand                           | Average      |          |          |          | 425.613     | 793.487 | 0.001260260          |
 689          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 690          + acpi-cpufreq-schedutil                          | 1            |          |          |          | 459.69      | 838.54  | 0.001192548          |
 691          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 692          + acpi-cpufreq-schedutil                          | 2            |          |          |          | 466.55      | 830.89  | 0.001203528          |
 693          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 694          + acpi-cpufreq-schedutil                          | 3            |          |          |          | 470.38      | 837.32  | 0.001194286          |
 695          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 696          + acpi-cpufreq-schedutil                          | Average      |          |          |          | 465.54      | 835.583 | 0.001196769          |
 697          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 698          + acpi-cpufreq-ondemand VS acpi-cpufreq-schedutil | Comprison(%) |          |          |          | 9.3810      | 5.3051  | -5.0379              |
 699          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 700          + amd-pstate-ondemand VS amd-pstate-schedutil     | Comprison(%) | 124.7392 | -36.2934 | 140.7329 | 1.1081      | 2.0661  | -2.0242              |
 701          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 702          + acpi-cpufreq-ondemand VS amd-pstate-ondemand    | Comprison(%) |          |          |          | 10.4665     | 7.5968  | -7.0605              |
 703          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 704          + acpi-cpufreq-schedutil VS amd-pstate-schedutil  | Comprison(%) |          |          |          | 2.1115      | 4.2873  | -4.1110              |
 705          +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
 706
 707 Reference
 708 ===========
 709
 710 .. [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
 711        https://www.amd.com/system/files/TechDocs/24593.pdf
 712
 713 .. [2] Advanced Configuration and Power Interface Specification,
 714        https://uefi.org/sites/default/files/resources/ACPI_Spec_6_4_Jan22.pdf
 715
 716 .. [3] Processor Programming Reference (PPR) for AMD Family 19h Model 51h, Revision A1 Processors
 717        https://www.amd.com/system/files/TechDocs/56569-A1-PUB.zip
 718
 719 .. [4] Linux Kernel Selftests,
 720        https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html